Don’t Make This Mistake: Painful Learnings of Applying AI in Security

The talk starts from a familiar problem: static analysis scans can return tens of thousands of findings, and triaging and fixing each one takes hours of developer time, so backlogs keep growing. The speaker frames the goal as automatically remediating first-party code so teams can keep shipping features while shortening the time between discovery and fix, at scale.

Early attempts used general-purpose chat models, prompted with a vulnerability description and the affected line. Results depended heavily on the prompt and the code. In a comparison across two intentionally vulnerable web applications and two commercial scanners, under a third of suggested fixes solved the problem cleanly, while many broke application logic, patched the wrong place, or deleted code. A live demo was disrupted by poor network connectivity. One SQL injection fix looked correct but broke the query because a numeric identifier was wrapped in quotes, and spotting that error required senior expertise. Reviewing thousands of model outputs one by one was shown to be too slow.

The alternative presented was a hybrid model. Deterministic, rule-based rewrites based on the code's syntax tree handle the transformation, while the model is limited to narrow sub-tasks such as locating user input or adding a null check. This gives repeatable output that is easier to review and scale, though each pattern takes effort to build. The takeaways: treat AI as a copilot rather than an autopilot, scope prompts tightly, validate every change, and expect developers to lose trust if bad fixes keep arriving. A tip on requesting machine-readable output was also shared.