Threat Modeling in the Age of AI
Frameworks and threat modeling methodologies adapted to identify and mitigate risks introduced by machine learning and AI systems.
Summary
Renowned security expert and author Adam Shostack takes the DEF CON 32 Main Stage to explore how threat modeling practices must adapt to the rise of artificial intelligence, large language models (LLMs), and automated reasoning systems.
[04:57] Shostack reviews the foundational methodology of threat modeling, reminding the audience that threat modeling is an early-stage engineering exercise designed to answer four essential questions before code is deployed: What are we building? What can go wrong? What are we going to do about it? Did we do a good job?
[10:08] The presentation examines the unique failure modes and attack surfaces introduced by AI systems, including prompt injection, data poisoning, model extraction, hallucination exploitation, and supply chain corruption in training pipelines. Shostack critiques the temptation to rely blindly on generative AI for security decision-making [20:58], warning against automation bias where engineers stop scrutinizing AI-generated threat models. He advocates for integrating structured AI threat modeling frameworks (like STRIDE for AI and MITRE ATLAS) into standard software development lifecycles.
(AI summary — Gemini Flash)
Related talks
Main Stage·50 min
Olivier Bilodeau · Estelle Ruellan
Demonstrates leveraging multimodal LLMs to analyze millions of infostealer infection screenshots and autonomously extract indicators of compromise.
Main Stage·40 min
Yisroel Mirsky
Side-channel traffic analysis attack that intercepts and reconstructs confidential AI assistant chat conversations across major platforms.
Main Stage·32 min
Alexandre Grattafiori · Ivan Evtimov · Ron Bitton
Methodologies, automated multi-turn adversarial agents, and benchmark tooling used by Meta's AI Red Team to test Llama 3.