Navigating the AI Minefield: Threat Modeling for Emerging AI Risks

Threat modeling for AI systems needs more than a standard checklist, because model outputs are less predictable and the risks reach beyond security into fairness and user trust. The talk opens by arguing that core principles such as authentication, least privilege, input validation, logging and monitoring still hold for AI. It then shows how STRIDE can be applied to AI components like data pipelines, models and APIs, using examples such as spoofing a fake model or tampering with training data.

The middle section surveys AI-specific threats: training data poisoning, model inversion, direct and indirect prompt injection, insecure output handling and overreliance on AI decisions. For each, the speaker pairs the attack with mitigations such as trusted data sources, access gating, output filtering, hardened instruction sets, sandboxed tool execution and human approval for critical actions. Resources from OWASP, MITRE ATLAS, NIST and Google's secure AI framework are mentioned, along with MAESTRO for multi-agent setups.

The closing case study applies this to a fictional banking assistant that feeds user feedback into training data and calls external APIs. The speaker identifies critical assets and trust zones, then maps threats and controls onto them. The main takeaway is that threat modeling should happen at design time, be revisited iteratively, and guide testing so the highest-priority threats are validated.