Incubated Machine Learning Exploits: Backdooring ML Pipelines Using Input-Handling Bugs

The talk introduces incubated ML exploits, a class of hybrid attack that pairs a classic system security flaw, specifically an input-handling bug, with a model backdoor. The speaker argues that ML security is often split between model research and system security, which leaves gaps where the two interact. Model files are parsed by many different tools, so a parser that reads a file differently from the reference implementation becomes an attack surface.

Much of the talk walks through examples organized by a language-theoretic taxonomy of input-handling bug classes. Examples include pickle deserialization that runs code during loading, bypasses of restricted unpicklers, and parser differentials in TorchScript and safetensors, where duplicate JSON keys let external parsers accept backdoored weights. Polyglot files that validate as two formats can hide a malicious model beside a benign one, and custom ONNX runtime operators offer another route into the architecture.

The closing guidance covers validating model integrity, keeping metadata well parsed, avoiding custom operators, separating weights from architecture, and using versioned formats with checksums and magic bytes. Practitioners should threat-model each pipeline stage, since impact depends on which component trusts which parser. The speaker frames machine learning as something that changes a system's overall security posture rather than a simple add-on.