Extracting the Ghost in the Machine

The talk opens with prompt injection, where crafted input gets a chatbot to ignore its rules and expose system prompts, internal records or private notes, including through tricks such as markdown image links that leak data. The speaker argues this matters because training data and hidden instructions can contain healthcare, financial and credential information.

The middle section moves from prompts to the models themselves. Data poisoning means slipping a small number of crafted or mislabelled samples into a training set, which can steer a model's behaviour; examples from assessments included a résumé screener and a code-sharing platform that could be made to return malicious code. Model inversion works in reverse, reconstructing faces and other private details from a model by querying it, even without direct access to its weights. Adversarial patches and subtle noise overlays push vision systems toward a different label, such as a toaster or a stop sign, and can be tuned to be nearly invisible to people. The speaker also described a hiding attack that suppresses a person's label, and a check-reading system whose amount could be misread.

The closing point is that these techniques are still early in research, and that the range of possible attacks is limited mainly by imagination. For practitioners, the takeaway is to treat training data, model outputs and any exposed query interface as part of the attack surface when assessing AI systems.