Killing AI Slop - A Multi Model Framework That Only Reports Findings It Can Prove

AI-generated vulnerability reports often fall apart under testing. They can cite endpoints that do not exist, exploit paths that cannot be reached, or flag behavior the application intends. The talk presents a pipeline that treats a finding as unconfirmed until it reproduces against the target. The first phase crawls the application with a headless browser, logs in with the supplied credentials, clicks through features, and collects endpoints, parameters and requests from pages and JavaScript files. The results are stored as a graph in a vector database, so agents can query how a user reaches a given function.

A threat-modeling step asks static and application-specific questions and turns the answers into threat classes. A router then spawns persona agents, each defined in a markdown file with an objective, inputs and a procedure for one vulnerability category. A validation council replays each candidate, checks it against scope, tests client-side issues in a real browser, and retries requests without session tokens. Confirmed findings are then checked for chaining, such as an access control flaw combined with stored XSS, to see whether the impact rises.

The main takeaway is that model output needs application context and an evidence gate before anything is reported. The speaker explains the goal as removing false positives, and says lighter models were used for routine steps because heavier models were slow. The talk is an overview with demo screenshots rather than a measured benchmark, so it does not quantify how many findings were filtered out.