Fixing My Fixing Talk: What We Got Wrong (and Right) About AI Auto-Remediation

Eitan Worcel revisits a talk from the previous year, where he argued that AI could not reliably fix code vulnerabilities. Six months of research at his company, Mobb, led him to a more nuanced view. He starts by noting how widely AI-generated code is now in use, and argues that tools should help fix the flaws it introduces. The central concern is reliability, since developers stop trusting tools that give inconsistent or inaccurate changes.

The core of the talk is a blueprint for generating fixes. Per-vulnerability prompts are built from public guidance and large collections of real vulnerable code. A first prompt separates true positives from false positives, with reasoning attached, so fixes are only generated for real issues. Fix prompts are then refined through repeated test runs and human review, with a second model used to critique them. Worcel shows a fix that looked correct but only partly sanitized input, and describes other failures, such as deleted file content and a removed return statement. Early prompts succeeded only 10 to 15 percent of the time, improving to roughly 70 to 90 percent after iteration.

He closes with practical lessons: validate every fix with syntax and style checks, keep changes minimal, use deterministic code for simple patterns to control cost, and keep a human in the loop. Accuracy matters more than coverage, and knowing when a tool fails matters most.