Backdooring Pickles: A Decade Later and Things Only Got Worse
ColdwaterQ demonstrates backdooring Python pickle files in machine learning models, weaponizing PyTorch and TensorFlow weights for silent remote code execution.
Summary
The Inherent Insecurity of Python Object Pickling
Python's pickle serialization format allows arbitrary code execution by design through the `__reduce__` method, a risk known for over a decade.
The Dangerous Intersection of Pickles and Machine Learning
ColdwaterQ reveals that modern AI/ML frameworks (PyTorch, Hugging Face, scikit-learn) routinely distribute model weights serialized as raw pickle files.
Crafting Stealthy Backdoored AI Models
The researcher demonstrates weaponizing popular pre-trained models on public repositories to execute reverse shells when loaded by data scientists.
Moving Toward Safe AI Model Serialization Formats
The author advocates for transitioning the AI industry to safe serialization formats like Safetensors and demonstrates automated pickle auditing tools.
Related talks
Main Stage·19 min
Tobias Diehl
Demonstrates how adversaries exploit data voids in LLM training corpora and search indexes to manipulate Microsoft Copilot answers and inject untrusted content.
Main Stage·19 min
C Parzian
C Parzian analyzes weaponized machine learning file formats, demonstrating arbitrary code execution vectors in widely deployed AI frameworks.
Main Stage·29 min
Ariel Herbert-Voss
Provides a structured adversarial ML red team methodology for attacking production AI systems beyond naive input perturbation.