# Kubernetes Access to other Clouds

## GCP

If you are running a k8s cluster inside GCP you will probably want that some application running inside the cluster has some access to GCP. There are 2 common ways of doing that:

### Mounting GCP-SA keys as secret

A common way to give **access to a kubernetes application to GCP** is to:

* Create a GCP Service Account
* Bind on it the desired permissions
* Download a json key of the created SA
* Mount it as a secret inside the pod 
* Set the GOOGLE\_APPLICATION\_CREDENTIALS environment variable pointing to the path where the json is.

{% hint style="warning" %}
Therefore, as an **attacker**, if you compromise a container inside a pod, you should check for that **env** **variable** and **json** **files** with GCP credentials.
{% endhint %}

### GKE Workload Identity

With Workload Identity, we can configure a[ Kubernetes service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) to act as a[ Google service account](https://cloud.google.com/iam/docs/understanding-service-accounts). Pods running with the Kubernetes service account will automatically authenticate as the Google service account when accessing Google Cloud APIs.

The **first series of steps** to enable this behaviour is to **enable Workload Identity in GCP** ([**steps**](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)) and create the GCP SA you want k8s to impersonate.

The **second steps** is to relate a K8s SA (KSA) with the GCP SA (GSA):

* Create the KSA (normally in the **annotations appear the email of the GSA**) and use it in the pod you would like to:

```yaml
# serviceAccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
    annotations:    
        iam.gke.io/gcp-service-account: $GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
    name: $APPNAME
    namespace: $NAMESPACE
```

* Create the KSA - GSA Binding:

```bash
gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:PROJECT-ID.svc.id.goog[$NAMESPACE/$KSA-NAME]" \
  $GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
```

Note how you are creating a binding and in the **member field you can find the namespace and KSA name**.

{% hint style="warning" %}
As an attacker inside K8s you should **search for SAs** with the **`iam.gke.io/gcp-service-account` annotation** as that indicates that the SA can access something in GCP. Another option would be to try to abuse each KSA in the cluster and check if it has access.\
From GCP is always interesting to enumerate the bindings and know **which access are you giving to SAs inside Kubernetes**.
{% endhint %}

This is a script to easily **iterate over the all the pods** definitions **looking** for that **annotation**:

```bash
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
    for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
        echo "Pod: $ns/$pod"
        kubectl get pods -n "$ns" -o yaml | grep "iam.gke.io/gcp-service-account"
        echo ""
        echo ""
    done
done
```

## AWS

### &#x20;Workflow of IAM role for Service Accounts:  <a href="#workflow-of-iam-role-for-service-accounts" id="workflow-of-iam-role-for-service-accounts"></a>

![](https://blogs.halodoc.io/content/images/2021/03/Group\_s3.png)

1. When you launch an application on kubernetes with `kubectl apply -f application-job.yaml`, the yaml manifest is submitted to the API server with the Amazon EKS Pod Identity webhook configured.
2. Kubernetes uses the service account set via serviceAccountName
3. Since the **service account has the annotation passed "eks.amazonaws.com/role-arn"** in `serviceaccount.yaml` the webhook injects the necessary environment variables (**AWS\_ROLE\_ARN** and **AWS\_WEB\_IDENTITY\_TOKEN**) and sets up aws-iam-token projected volumes.
4. When application calls out to s3 to do any s3 operations the AWS SDK we use in the application code base performs STS **assume role** with web identity performs assume role that has s3 permissions attached. It receives temporary credentials that it uses to complete the S3 operation.

(You can find an example of this configuration [here](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/))

Just like with GCP an **annotation** is needed to relate the KSA with the IAM role:

```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws-cn:iam::ACCOUNT_ID:role/IAM_ROLE_NAME
```

{% hint style="warning" %}
As an attacker, if you can enumerate a K8s cluster, check for **pods with that annotation** as you could manage to **escalate to AWS**.

Moreover, if you are inside a pod, check for env variables like **AWS\_ROLE\_ARN** and **AWS\_WEB\_IDENTITY\_TOKEN.**
{% endhint %}

This is a script to easily **iterate over the all the pods** definitions **looking** for that **annotation**:

```bash
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
    for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
        echo "Pod: $ns/$pod"
        kubectl get pod "$pod" -n "$ns" -o yaml | grep "eks.amazonaws.com/role-arn"
        echo ""
        echo ""
    done
done
```



## References

* [https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)
* [https://blogs.halodoc.io/iam-roles-for-service-accounts-2/](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/)