119 lines
5.7 KiB
Markdown
119 lines
5.7 KiB
Markdown
# Kubernetes Access to other Clouds
|
|
|
|
## GCP
|
|
|
|
If you are running a k8s cluster inside GCP you will probably want that some application running inside the cluster has some access to GCP. There are 2 common ways of doing that:
|
|
|
|
### Mounting GCP-SA keys as secret
|
|
|
|
A common way to give **access to a kubernetes application to GCP** is to:
|
|
|
|
* Create a GCP Service Account
|
|
* Bind on it the desired permissions
|
|
* Download a json key of the created SA
|
|
* Mount it as a secret inside the pod 
|
|
* Set the GOOGLE\_APPLICATION\_CREDENTIALS environment variable pointing to the path where the json is.
|
|
|
|
{% hint style="warning" %}
|
|
Therefore, as an **attacker**, if you compromise a container inside a pod, you should check for that **env** **variable** and **json** **files** with GCP credentials.
|
|
{% endhint %}
|
|
|
|
### GKE Workload Identity
|
|
|
|
With Workload Identity, we can configure a[ Kubernetes service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) to act as a[ Google service account](https://cloud.google.com/iam/docs/understanding-service-accounts). Pods running with the Kubernetes service account will automatically authenticate as the Google service account when accessing Google Cloud APIs.
|
|
|
|
The **first series of steps** to enable this behaviour is to **enable Workload Identity in GCP** ([**steps**](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)) and create the GCP SA you want k8s to impersonate.
|
|
|
|
The **second steps** is to relate a K8s SA (KSA) with the GCP SA (GSA):
|
|
|
|
* Create the KSA (normally in the **annotations appear the email of the GSA**) and use it in the pod you would like to:
|
|
|
|
```yaml
|
|
# serviceAccount.yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
annotations:
|
|
iam.gke.io/gcp-service-account: $GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
|
|
name: $APPNAME
|
|
namespace: $NAMESPACE
|
|
```
|
|
|
|
* Create the KSA - GSA Binding:
|
|
|
|
```bash
|
|
gcloud iam service-accounts add-iam-policy-binding \
|
|
--role roles/iam.workloadIdentityUser \
|
|
--member "serviceAccount:PROJECT-ID.svc.id.goog[$NAMESPACE/$KSA-NAME]" \
|
|
$GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
|
|
```
|
|
|
|
Note how you are creating a binding and in the **member field you can find the namespace and KSA name**.
|
|
|
|
{% hint style="warning" %}
|
|
As an attacker inside K8s you should **search for SAs** with the **`iam.gke.io/gcp-service-account` annotation** as that indicates that the SA can access something in GCP. Another option would be to try to abuse each KSA in the cluster and check if it has access.\
|
|
From GCP is always interesting to enumerate the bindings and know **which access are you giving to SAs inside Kubernetes**.
|
|
{% endhint %}
|
|
|
|
This is a script to easily **iterate over the all the pods** definitions **looking** for that **annotation**:
|
|
|
|
```bash
|
|
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
echo "Pod: $ns/$pod"
|
|
kubectl get pods -n "$ns" -o yaml | grep "iam.gke.io/gcp-service-account"
|
|
echo ""
|
|
echo ""
|
|
done
|
|
done
|
|
```
|
|
|
|
## AWS
|
|
|
|
###  Workflow of IAM role for Service Accounts: <a href="#workflow-of-iam-role-for-service-accounts" id="workflow-of-iam-role-for-service-accounts"></a>
|
|
|
|
|
|
|
|
1. When you launch an application on kubernetes with `kubectl apply -f application-job.yaml`, the yaml manifest is submitted to the API server with the Amazon EKS Pod Identity webhook configured.
|
|
2. Kubernetes uses the service account set via serviceAccountName
|
|
3. Since the **service account has the annotation passed "eks.amazonaws.com/role-arn"** in `serviceaccount.yaml` the webhook injects the necessary environment variables (**AWS\_ROLE\_ARN** and **AWS\_WEB\_IDENTITY\_TOKEN**) and sets up aws-iam-token projected volumes.
|
|
4. When application calls out to s3 to do any s3 operations the AWS SDK we use in the application code base performs STS **assume role** with web identity performs assume role that has s3 permissions attached. It receives temporary credentials that it uses to complete the S3 operation.
|
|
|
|
(You can find an example of this configuration [here](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/))
|
|
|
|
Just like with GCP an **annotation** is needed to relate the KSA with the IAM role:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
annotations:
|
|
eks.amazonaws.com/role-arn: arn:aws-cn:iam::ACCOUNT_ID:role/IAM_ROLE_NAME
|
|
```
|
|
|
|
{% hint style="warning" %}
|
|
As an attacker, if you can enumerate a K8s cluster, check for **pods with that annotation** as you could manage to **escalate to AWS**.
|
|
|
|
Moreover, if you are inside a pod, check for env variables like **AWS\_ROLE\_ARN** and **AWS\_WEB\_IDENTITY\_TOKEN.**
|
|
{% endhint %}
|
|
|
|
This is a script to easily **iterate over the all the pods** definitions **looking** for that **annotation**:
|
|
|
|
```bash
|
|
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
echo "Pod: $ns/$pod"
|
|
kubectl get pod "$pod" -n "$ns" -o yaml | grep "eks.amazonaws.com/role-arn"
|
|
echo ""
|
|
echo ""
|
|
done
|
|
done
|
|
```
|
|
|
|
|
|
|
|
## References
|
|
|
|
* [https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)
|
|
* [https://blogs.halodoc.io/iam-roles-for-service-accounts-2/](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/)
|