Kates’ Pot: Finding Attacks Against Kubernetes Deployments

### Overview In this talk, Matthias Meitinger—Community Manager for StackRox and employee at Red Hat Advanced Cluster Security (part of OpenShift)—presents "Kates' Pot," an open-source medium-interaction Honeypot designed to detect and analyze attacks targeting exposed Kubernetes deployments. Meitinger explains that while Kubernetes is widely deployed, its operational and architectural complexity creates significant configuration challenges. By emulating a small developer or edge Kubernetes environment (specifically Minikube), the project captures internet-wide scans, exploitation attempts, and reconnaissance activities over a 60-day observation window.

### Key Topics - **Honeypot Classification and Architecture:** Meitinger details low, medium, and high-interaction honeypots, noting that while low-interaction setups capture credentials on minimal hardware and high-interaction environments proxy real systems at higher liability and resource costs, Kates' Pot uses a medium-interaction design written in Go to mimic original gRPC and API interfaces. - **Emulated Kubernetes Surfaces:** The Honeypot emulates both the Kubernetes API Server and the node-level kubelet, capturing client tooling interactions such as `kubectl get pods`, `kubectl get nodes`, and offensive utilities like `kubeletctl`. - **Threat Landscape Observations:** Analyzing 60 days of traffic via Splunk dashboards revealed very few attacks specifically tailored to Kubernetes. Instead, traffic predominantly consisted of broad web exploitation (e.g., Spring Framework, Tomcat bypasses), opportunistic read-only reconnaissance, crypto mining, botnets, and cloud metadata access attempts. - **Kubernetes Hardening & Best Practices:** A core focus is applying native Kubernetes security primitives—such as Role-Based Access Control (RBAC), Network Policies, Admission Controllers, and Pod Security Admission—before purchasing external security tooling.

### Technical Details - **Implementation:** Kates' Pot is compiled in Go to natively mirror Kubernetes interfaces without running real workloads, replaying recorded JSON API responses pulled from an actual cluster. This prevents attackers from compromising a real cluster while maintaining realistic responses for CLI tools. - **Logging and Ingestion:** The Honeypot collects user agents, raw API requests, and container execution or creation attempts, outputting structured JSON logs piped into a Splunk dashboard for ASN and path visualization. - **Observed Network and Scanner Activity:** Commercial and internet scanners observed included Shodan, Palo Alto Networks, Censys, Baidu, and an entity called GDN Plus. Network origins included significant traffic from DigitalOcean, Microsoft ASNs, AS202984, and clusters of kubelet traffic originating from Prague (Czechia). Traffic from OVH dropped dramatically following their French data center fire. - **Exploitation Nuances:** Few attackers deployed custom Kubernetes payloads, which Meitinger attributes to the Honeypot emulating a small Minikube cluster on a small German hosting provider rather than prominent cloud provider IP ranges (like GKE or AKS), as well as attackers favoring simpler targets like exposed Docker sockets.

### Takeaways - **Enforce Native Controls First:** Prioritize built-in mechanisms over complex third-party tools by configuring RBAC for granular permissions, implementing internal Network Policies for pod-to-pod firewalling, and utilizing Pod Security Admission (and retiring deprecated Pod Security Policies) to block privileged containers and host path mounting. - **Protect Critical Ports:** Never expose the kubelet or API server directly to the public internet without proper authentication and authorization. - **Maintain Cluster Hygiene:** With Kubernetes issuing releases three times per year, clusters must stay up to date to prevent forced cloud provider upgrades and unsupported legacy versions. - **Visibility:** Turn on Kubernetes audit logging, route logs to a SIEM, and maintain accurate workload inventory to quickly differentiate legitimate workloads from unauthorized containers.