Safeguarding Machine Learning Systems: A Comprehensive Security Plan

The talk opens by comparing the current state of machine learning security with where web application security stood two decades ago, when injection flaws were poorly understood and tooling was immature. The speaker walks through the engineering side of an ML system: preparing data, choosing an algorithm, training to reduce prediction error, evaluating the model, deploying it as a file behind an API, and operating and updating it.

The middle section surveys attacks against these systems. These include evasion of classifiers such as spam filters, model extraction through repeated queries, backdoors and poisoned training data carried in pre-trained models from public hubs, and malware hidden in model files. Prompt injection, including indirect injection through uploaded documents, is covered as a newer category. A chatbot case in which a company was held to a promise its bot had made is used to show the business impact. The speaker proposes a framework with five areas: data, model, platform, compliance and human security. Each area has stated goals, requirements and guidance, and the speaker points to an open-source project that collects policies and tool lists.

The closing part covers privacy-preserving techniques, including differential privacy, federated learning, homomorphic encryption with its performance cost, secure multi-party computation and synthetic data. The talk ends with a phased roadmap: draft policy and run red teaming in the first three months, raise developer awareness, then evaluate privacy techniques and threat model ML systems over the following months before automating. Practical takeaway: understand the system thoroughly before threat modeling it, and use hands-on labs to learn the attacks.