Implementing Application Whitelisting with Google Santa and Upvote (Part 2 of 2)

In the first part of this series, we described a real-life example of implementing application whitelisting inside of Praetorian and the challenges that were overcame. Readers learned more about application whitelisting, our process for evaluating a solution, and how we developed a plan for deployment. In part 2, we will provide technical guidance on how you can deploy Google Santa and Upvote in your organization.

Read: Implementing Application Whitelisting with Google Santa and Upvote (Part 1 of 2)

Rolling out our application whitelisting solution

First, we conducted multiple studies in order to understand user workflows so we could proactively whitelist commonly used applications. This allowed us to dramatically decrease the number of applications users had to whitelist in order to get back into their normal workflow.

There are three components to our deployment:

• The Upvote application which runs in Google App Engine.
• The Santa component which enforces application whitelisting on endpoints.
• An MDM solution used to manage both device identity and deployment of the Santa application.

Deploying the Upvote server

We found deployment fairly easy compared to most products. The deployment guide provided with Upvote will give you just about everything you need. However, if you are unfamiliar with Google App Engine you may run into issues. We found that using a docker container to build the application provided a consistent environment and minimized a number of potential deployment headaches.

These are the steps we took to deploy the application:

1. Download Docker and install it
2. Make a directory called upvote-deploy on your build workstation
3. Make a new Dockerfile in that folder with the following contents

		FROM ubuntu:16.04&nbsp#first lets get bazel setupRUN apt-get updateRUN apt-get install -y openjdk-8-jdk curl git nanoRUN echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" |  tee /etc/apt/sources.list.d/bazel.listRUN curl https://bazel.build/bazel-release.pub.gpg |  apt-key add -RUN apt-get update &&  apt-get install -y bazel&nbsp#Get gcloud bins&nbspRUN echo "deb http://packages.cloud.google.com/apt cloud-sdk-xenial main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.listRUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg |  apt-key add -RUN apt-get update && apt-get install google-cloud-sdk -yRUN mkdir /app&nbsp#get python deps for upvote&nbspRUN apt-get install python-pip python-dev -yRUN pip install six&nbspWORKDIR /app&nbspRUN git clone https://github.com/google/upvote.git	

After that, continue with the following:

1. Build the container with: docker build -t upvote_build ./
2. Run the build container with: docker run -it --entrypoint bash upvote_build
3. Follow the steps here: https://github.com/google/upvote/blob/master/docs/setup.md
4. Create a new Google Cloud Project
5. Set to project name in the docker container with: export PROJ_ID="my-proj-id", where my-project-id is the project idea created in the google cloud console.
6. Log into the gcloud tool in the container by running: gcloud init
7. Run ./init_project.sh
8. During our deployment we got the following error and spent a long time trying to debug it, however, this is normal, when you are ready to test deploy run this command
9. To deploy run: bazel run upvote/gae:monolith_binary.deploy
10. Now you need to customize the deployment. Make sure you edit the upvote/gae/settings.py file
11. nano upvote/age/settings.py
12. Redeploy via bazel run upvote/gae:monolith_binary.deploy
13. Once you have it up, go to it via https://PROJECT_NAME.appspot.com and login with your email that you defined in the settings.py

Access Security (Extra Credit)

In addition, the Upvote application can be placed behind an identity aware proxy (e.g., Google BeyondCorp style proxy) to control access to the application with both user and device authentication.

Deploying Santa

At Praetorian, we believe all aspects of our IT environment should be code. We use a configuration manager called Chef to manage the deployment of software on endpoint systems. With Chef in our stack, getting the binary service installed was as simple as writing a chef script and deploying that to our internal endpoints.

		cookbook_file '/opt/praetorian/santa.dmg' dosource 'santa-0.9.31.dmg'mode '0700'owner 'root'group 'wheel'end&nbspdmg_package 'install santa' doaccept_eula truefile '/opt/praetorian/santa.dmg'type 'pkg'not_if {File.exists?('/usr/local/bin/santactl')}end	

Once Santa is installed across the environment, the final piece is to tell Santa where to check-in for whitelisting updates. This leads us to configuration of the MDM.

Configuring our MDM

The MDM provider will supply this component of the toolchain. We use SimpleMDM, however, any macOS MDM provider that accepts .mobileconf files will work. First, we needed to deploy a custom .mobileconf to all our endpoints. This can be done with or without Santa installed.

SimpleMDM has a nice feature called attribute support. You will need to make sure you fill out the MachineOwner key with the users’ email address. Without this, Upvote would not be able to customize the whitelist for the endpoints on a per user basis. We will be releasing the cloud functions that allow us to marry Google with SimpleMDM and the device identity in a future blog post, however, this step can be done manually.

— GIST https://gist.github.com/myover/90a4197b133b6ed616be1dfb37e106d9.js —

Finally, we need to whitelist our Santa kernel extension. This can be achieved by a .mobileconfig or using the native tools built into your MDM provider.

User experience

By default, Santa will check in for a new whitelist every 10 minutes. If a user upvotes something, they may want to force a Santa sync using sudo santactl resync. This may work for engineers, but not for business users. We developed a wrapper script using the Hello IT menu to make syncing the whitelist more user-friendly.

A sample MDM profile that provides a button to sync the whitelist.

— GIST https://gist.github.com/myover/bd1c8d7ee671af3b0d8c8faec17b429b.js —

And the script is as follows:

— GIST https://gist.github.com/myover/0c7f7e49b37d1312c6f55dffa7d4e5ce.js —

User Study

So after we got a deployment going, we started determining the best method of training users and soliciting feedback from people we suspected this change would highly impact and potentially disrupt. During this period we worked one-on-one with users to determine what would negatively impact their workflow the most. Using this information we pre-whitelisted applications that were running on users’ workstations. As part of our deployment, we put all workstations into monitoring mode and would only enforce an endpoint after that user completed training. Monitoring mode made it easier to pre-whitelist applications for users to remove some of the burden of implementing application whitelisting.

From the post-deployment survey, we saw a trend where most users only had to whitelist one or two apps and their normal workflow returned to normal in under 1 hour.

From the user study we determined that user training was required to further minimize negative impact to the end user. Because of this we decided to only enable the control after the user was properly trained and also ensured the training spanned a wide audience, from technical- to business-groups. Our training materials included the following aspects:

• Overview of the technology and terms deployed
• Business case and aspects
• Technical deployment information

Future Projects

We noted several future projects worth exploring, which include:

• Detecting bypasses of Santa
• Automating endpoint exclusion of the control
• Integrating Device Identity into Santa

Conclusion

Whitelisting is not a silver bullet, however, it can provide an additional control to help pigeonhole attackers into predictable patterns of behavior in order to increase the chances that a potential attack will be blocked, or if an attacker is able to execute a payload, they will be restricted to a set of well-known tactics, techniques, and procedures that are all monitored and baselined for anomalies. We found that user training and explaining the motivation behind application whitelisting provided considerable buy-in from end users.

Using a system such as Upvote, we found that users felt they were still in control of application whitelisting and users themselves have had the largest impact in the solution’s success.