Out of Control: Promise Theory and the Future of Security Agents

The talk argues that security teams are repeating an old habit by managing AI agents like guards or factory workers that wait for orders. The speaker warns that this approach yields brittle systems and misaligned incentives. In its place, the talk proposes Promise Theory, a model from Mark Burgess in which autonomous agents state what they will do, and trust grows as they keep or break those commitments over time.

Drawing on experience building an agentic code review product, the speaker described the problems that shaped the design: repeatable results, high accuracy, fast feedback, and limited visibility into the code around a change. The pipeline splits work among context-gathering, planning, verification, and exploitability agents, with specialized agents for each vulnerability class and models matched to each job. A benchmark across 26 vulnerabilities in four languages compared detection rates among several tools. An audience exercise had attendees name a security tool as an agent and list its promises and dependencies. The speaker also walked through the OWASP LLM Application Top 10, from prompt injection to unbounded consumption, and described a cost spike of roughly $2,500 caused by an agent searching a codebase without limits.

The closing point is that risk has become contextual, so analysis should weigh code intent, application design, and the operating environment rather than matching patterns alone. Practitioners should keep each agent's scope narrow, measure how reliably agents deliver on their commitments, and keep a developer in the loop for automated fixes.