Evading LLM Detection

The speaker, from a crypto exchange's security team, explains why supply chain attacks on npm and PyPI packages need more than signature rules. Their team used an AI model to read suspicious code and reason about intent, which caught obfuscated payloads, including one in a package with billions of weekly downloads. Because the model judges behaviour rather than matching known patterns, it could flag hidden logic that scanners had missed.

The core of the talk is an internal capture-the-flag exercise in which engineers tried to push malicious code past an LLM check placed in front of a CI build. The first bypasses exploited the model's limits: flooding it with so many files that a tampered file was skipped, hiding a payload inside an image file, and planting prompt-injection text in comments. Each fix required large redesigns, which led the team to a multi-agent setup with an orchestrator and a second verifying agent, plus a separate binary-analysis step. A later bypass used Git submodules that pointed to earlier commits, and the team had to fall back on hardening the build environment with conventional controls.

The takeaways are practical. LLM scanning avoids maintaining large signature databases, but it brings false positives, inconsistent output, and real cost. The speaker recommends starting with one agent on a narrow focus, combining it with fast tools such as Semgrep, using cheaper models where possible, and keeping model reasoning out of reach of anyone who could exploit it.