"Secure AI" is 20 years old

Sven Cattell opens with the claim that machine learning security is much older than the current generative AI wave, pointing to a spam-filter bypass documented in 2004 as an early example. He argues that many people now calling themselves AI red teamers lack this history, and that some work labeled red teaming, such as brand-safety filtering or bias testing, has no real adversary behind it.

The middle of the talk draws lessons from years of production spam and malware classifiers. Models are never perfect, attackers keep finding inputs outside the training distribution, and features that users can easily manipulate tend to fail. Cattell describes a malware detector whose allowlist was bypassed by appending data to a trusted game binary, and a social network that limited exposure by training models on random feature subsets and redeploying them frequently. He also treats adversarial-example defenses with skepticism and argues that data poisoning is rarely economical against large production systems.

He closes with a framework: assume a model will eventually be bypassed, ask whether a control buys time or speeds response, and invest in detection and fast redeployment. For language models, he suggests treating prompt injection and information leakage like the older problems, applying classifiers and layered controls, and learning from the people who already attack these systems.