AI in a Minefield: Learning from Poisoned Data

The talk opens with a short survey of AI risks, including deepfakes, discrimination in automated decisions, and attacks on models themselves such as evasion and training data leakage. It then focuses on data poisoning, where an attacker slips crafted or mislabeled samples into training so a model learns the wrong behavior. Simple averaging examples show how one malicious input can carry outsized weight, and real cases include a 2018 attack on a Gmail spam filter.

The middle section turns to web application firewalls, which learn a baseline profile of API traffic from sources that cannot be fully trusted. Mitigations include filtering suspicious origins, capping how much any single IP, user agent, or time window can influence learning, and comparing successive models for drift. The speaker then presents a streaming-friendly method built on boolean facts: a value is learned only after it appears across enough distinct entities, so memory scales with the number of properties rather than the volume of data. The approach covers parameter types, character sets and numeric ranges, and extends to decision trees and ensembles.

The closing point is that no single fix eliminates data poisoning, but threshold-based learning can raise the cost for attackers and limit damage if tailored to each problem. Anyone building models on user-supplied data should treat that data as an attack surface.