Living off Microsoft Copilot

The talk treats Microsoft Copilot as a target inside an enterprise tenant, using illustrative characters such as a CISO, a security engineer and an employee to frame the risk. The speaker explains that the assistant can read far more organizational data than its users see directly, and that sensitivity labels and reference checks do not cover every path. Early attempts to extract data through generated images, arbitrary URLs or code execution were blocked, so the research moved to other inputs.

The central technique is retrieval poisoning. An attacker sends an email or a Teams message from an external tenant that carries hidden instructions and fabricated retrieval snippets. When an employee later asks a routine question, Copilot folds that content into its answer. In the demonstrated case, the answer showed changed bank details while still citing the legitimate source file, which made the manipulation hard to spot. The talk also showed reconnaissance of what the assistant knows, and the generation of convincing spear-phishing emails from existing thread content.

The takeaways are that, for a language model, data effectively becomes instructions, and that jailbreaks are unlikely to disappear. Defenders should treat AI features as experimental, review the references they produce and assume that English-focused safeguards may be uneven. The speaker noted that some requests worked better in other languages, such as Russian.