Testing API Business Logic With AI Agents - What We Got Wrong First

This talk looks at automated testing of business logic flaws in APIs, such as broken object-level authorization and abuse of legitimate workflows. These bugs are hard for conventional scanners to catch because they need an understanding of how an application is meant to behave. The speaker argues that they are becoming more common as AI-generated applications and endpoints multiply. Broad scanners cover a lot of ground but lack depth, and manual testers have depth but cannot scale. Early experiments that pointed a general-purpose model at an application found one or two valid issues and then stopped.

The central lesson was that a bare model is lazy and needs a surrounding harness. The team built a cycle in which a crawler keeps discovering endpoints, an orchestrator hands them to specialized sub-agents, and findings move into a validation stage. Against a deliberately vulnerable open-source fork, this design gave far more API coverage and more business logic findings than the model alone. Validating these flaws is harder than validating reflected injection, so the team added an adversarial agent whose job is to disprove each finding and cut false positives.

The closing point is that domain context decides whether something is a flaw at all. A coupon marked as internal is only a problem if someone understands the business, which a deterministic check cannot judge. For practitioners, the takeaway is that agentic testing needs planning, memory, and verification around the model, along with retesting rather than matching identifiers. The talk is candid about what went wrong, which makes it useful for anyone building similar pipelines.