Agents Under Siege - Live Attacks from RAG to Tool Calls to Protocols

Agentic AI systems increasingly turn retrieved documents and model plans into tool calls that change real systems, yet many defenses still focus only on the user's prompt. The talk argues that the risky moment is the "second prompt": the machine-level instruction a model generates for a tool after planning. Aviral Srivastava frames this as the point where a read request can become a write, and where security teams need visibility and validation.

The talk walks through three scripted demonstrations against a local orchestrator and simulated finance, data and plugin services. In the first, a poisoned document retrieved for a read-only summary request is grafted into the plan as a vendor-term change, and a required ticket precondition plus an allowed-value list stop it. The second shows argument drift, where error text suggesting a different call during retries turns a read into an unregistered write, caught by schema linting, a drift comparison and write approval checks. The third covers a malicious MCP plugin that returns a correct summary while sending data to an external endpoint, addressed through plugin attestation, declared scopes and egress allow-listing.

The closing portion covers what to log for each model-generated action, such as tool names, arguments, retry counts, retrieval sources and plugin identity. It also covers pre- and postconditions, schema-aware linting, metrics such as the unsafe tool call ratio, AI-specific incident response evidence, and a 30- and 60-day hardening plan.