Wrestling with a Python - Escaping Copilot Studio's AI Guarded Sandbox

The talk starts with Microsoft Copilot Studio, where agents can run Python through a code interpreter. The presenters treat that interpreter as a target. Behind it sit a Windows container on Azure Service Fabric, a Python sandbox, and a small language model that decides whether each piece of code is safe to run. They picked the smallest model in the settings, expecting it to be the easiest to get past.

Early probes used module introspection and file-reading helpers to reach sandbox internals and TLS material. The guardrail turned out to be non-deterministic: the same payload sometimes passed and sometimes failed. Its API responses also echoed its security reasoning, which gave the team feedback for refining payloads. Walking Python's method resolution order got around the dunder filters. The team then found a hard-coded administrator credential in a related Dataverse plugin environment that also appeared in the Copilot containers, and it cracked quickly. The haul included certificates, private keys, around 75 environment variables, and application source, along with command execution as a container user.

Microsoft first rated the reports low severity. The speakers said the escape still worked in some environments after the tickets were marked fixed, and they released their tooling. The takeaways: sandboxing untrusted code is hard, LLM-based guardrails are fallible, and organizations should keep sensitive data out of agent sandboxes.