This Wasn't AI Generated - Principles for Breaking Generative Watermarks

The speakers examine image provenance watermarks, focusing on Google's SynthID, which Gemini checks through a verification tool. They begin with baseline tests using simple distortions such as grayscale conversion, flips and cropping. The detector resisted most of these, but screenshots passed through it reliably, and the team says the detector struggles with cropping and borders.

The main attack is regeneration: a watermarked image is noised and then lightly denoised with an off-the-shelf diffusion model. About ten steps were enough to drop Gemini's detection rate to roughly 20 percent on their set of about 100 images, while the picture stayed mostly intact. To avoid querying the live service repeatedly, the team built a surrogate detector from pairs of photos and Nano Banana edits in Apple's Pico-Banana dataset, which carry the watermark implicitly. A convolutional classifier trained on these pairs ran on a CPU in about 27 milliseconds and reached roughly 90 percent accuracy on held-out data, so attack variants could be screened offline first.

The speakers also report that removing the watermark changed how Gemini described images, sometimes producing confident claims about fictional scenes. They suggest this could matter for agent confused-deputy attacks, retrieval poisoning and facial recognition. Their takeaways favor layered provenance, common cross-vendor standards and content-level reasoning over any single watermark. They note that human quality checks were limited.