Security conscious companies prevent arbitrary egress (connections to the Internet) from within a secure datacenter or cloud environment as a means to prevent exfiltration or remote command and control by attackers. However, if the secured zone uses public cloud resources such as Storage or Pub/Sub, it is easy to circumvent this network control.
We present this attack and defend scenario for the GCP cloud, however, similar techniques and remediations apply to all major cloud vendors. Similarly, we used a compromised Kubernetes pod as the foothold, but this would apply equally to a compromised VM, App Engine or serverless Cloud Function with the appropriate Service Account permissions. Finally, this exfiltration technique could be applied to other services such as Pub/Sub or BigQuery.
On a recent engagement, we gained the ability to execute code on a pod which we compromised through a SQL injection vulnerability. With the SQL injection, we could write pickled python objects to a table in a database and those objects would be unpickled and executed by a different pod. The customer was using a Private Kubernetes GKE cluster and had restricted all egress traffic to pass through a network proxy which denied general access to the Internet. This prevents standard exfiltration of data from the pod. However, the pods had permissions to write to GCP Storage Buckets, and therefore, the storage.cloud.google.com URL was whitelisted.
Since GCP uses the same domain root for all customer Storage Buckets, all the excellent networking best practices employed by the customer were foiled by an attacker who could simply copy any customer data to their own GCP Bucket or download needed malware into the customer network by first copying any desired resource into their own GCP Bucket. Similar techniques could be used if Pub/Sub or other storage services were granted to a GCP compute resource. How can we defend against this?
GitHub Project: Creating VPC Service Controls and simulating the attack →
GitHub Project: Can We Use Bucket ACLs or Policy Conditions (Private Beta)?
A first attempt at preventing exfil may be to use Bucket ACLs. Bucket ACLs are a fine-grained way of controlled access to storage buckets and the individual objects in them. While Cloud IAM policies apply to all object in a bucket, ACLs can be applied to individual objects. The bucket ACLs created for a project only apply to the project’s own Storage Buckets, not an attackers’. You might try restricting the permissions attached to the Service Account. Restricting access to the resource is currently only available in beta preview.
{"bindings": [{"role": "roles/storage.objectEditor","members": "service_account:$SERVICE_ACCOUNT@$PROJECT_NUMBER.iam.gserviceaccount.com","condition": {"title": "only_allow_company_buckets","description": "Only permit reads and writes to a specific company bucket","expression": resource.service == "google.cloud.storage" &&"resource.name.startsWith("projects//example-co-bucket/ ")} }]}
However, it is possible for an attacker to make a pubic writeable bucket which allows any client to upload.
gsutil acl ch -u AllUsers:RW gs://attackers-exfil-bucket/
Foiled.
The Solution
Use VPC Service Controls to restrict access to specific GCP resources VPC Service Controls allow users to define a security perimeter around Google Cloud Platform resources such as Cloud Storage buckets, Bigtable instances, and BigQuery datasets to constrain data within a VPC and help mitigate data exfiltration risks. This method requires a GCP Organization account.
The user performing the following actions must have these roles:
- roles/accesscontextmanager.policyAdmin
- roles/dns.admin
- roles/compute.network.*
The steps are:
- Create a Service Perimeter around the GCP services you want to control
- Create a Private Google Access subnet for restricted.googleapis.com restricted.googleapis.com must resolve to 199.36.153.4/30
- Modify the DNS for the VPC in the service perimeter adding *.googleapis.com as a CNAME to restricted.googleapis.com
The diagram below shows a VPC deployment with a service perimeter preventing exfiltration by GCP Services.
Detailed steps for creating VPC Service Controls and simulating the attack are available on Github.
