Wiring the Harness Orchestrating Frontier Models for Vuln Hunting at Scale
Where
Packet Hacking Village
DEF CON 34
Duration
42 min
Presents a five-stage harness that pairs frontier models with scoping, retrieval, deduplication and exact-tool checks to cut false positives in LLM vulnerability hunting.
Summary
Intel's offensive research team set out to scan hundreds of repositories, including firmware, containers and low-level C and C++ code, on a tight budget with early access to a frontier model. The speakers argue that asking a model to "find vulnerabilities" mostly relocates the needle, because it produces a large volume of plausible reports that reviewers must investigate one by one. Their answer is to treat the model as one engine inside an engineered system, where scripts, retrieval and deterministic tools handle the mechanical work.
The talk walks through a five-stage workflow. The first stage scopes the problem using threat models, specifications, diagrams and build artifacts, then compresses reachable code into bounded candidate paths. The second stage gives each path to a model for hunting, with cheaper models used for routine audit checks. Deduplication follows, using scripts and a light model to merge overlapping findings. Verification then re-grounds each claim in the actual sources and uses exact tools such as an SAT solver to reject impossible candidates. Patterns that survive review can be encoded as static rules. The final stage tests claims against emulators or real targets and feeds confirmed results back into scoping. The speakers also cover common anti-patterns, such as hand-chunking code and scanning every function, and report that the frontier model accounted for most of the cost.
The main takeaway is that a claim must earn its way forward. Practitioners should scope before scanning, route simple tasks to smaller models, ground every claim in source and build evidence, challenge reports before sending them to humans, and save what verification teaches for the next run.
Related talks
Bug Bounty Village·27 min
Bogdan Steele · Arnav Garg
Presents MPG, a framework that mutates confirmed indirect prompt injection payloads to measure how many variants still bypass an AI assistant's safeguards.
IoT Village·42 min
Larry Pesce
Shows how an AI agent driven by an IoT pentest playbook found and chained flaws in a webcam, and what humans must still validate.
Main Stage·47 min
Ji'an Zhou · Lei Lu
Shows how a PyTorch deserialization flaw, reached through AI inference APIs, turns a local model-loading bug into remote code execution.


