#498 Learn What You Want to Unlearn Unlearning Inversion Attacks against Machine Unlearning Hongsh

Machine Unlearning Basics

The speaker contrasts standard machine learning with machine unlearning, where specific training samples must be removed for reasons such as the right to be forgotten. Unlearning produces two model versions—the original and the unlearned model—which creates a new privacy attack surface beyond traditional membership inference.

Unlearning Inversion Attacks

The research asks whether confidential information within unlearned data can still be inferred. With white-box access to both models, attackers subtract parameters to recover gradient information and run optimization to invert features of removed samples. With black-box access, probing samples with balanced class confidence can reveal the unlearned label via the class whose confidence drops most.

Experimental Findings

Approximate unlearning leaks far more feature information than exact retraining-from-scratch unlearning, which benefits from training randomness. In multi-sample unlearning, full feature recovery is harder but partial inversion remains possible. Label inversion succeeds for both exact and approximate methods across single- and multi-class removal scenarios.

Defenses and Limitations

The team evaluated obfuscation, model pruning, and fine-tuning on additional data to blur differences between model pairs, but each defense caused unacceptable privacy-utility tradeoffs. They call for stronger post-unlearning protections against inversion attacks.