Membership inference attacks from first principles

### Overview This presentation explores membership inference attacks against machine learning models from first principles, aiming to determine whether a specific example was included in a target model's training set. The speaker explains that while directly computing the probability distribution of high-dimensional neural network weights is intractable, attackers can instead evaluate model loss on a specific input as a scalar value. By re-evaluating membership inference through per-example parametric distributions rather than single global thresholds, the authors present an attack that is both conceptually grounded and significantly more effective in high-precision regimes.

### Key Topics - **Foundations of Membership Inference**: Framing membership inference as a statistical test comparing the probability that a model was trained on an example versus not trained on it. - **Limitations of Weight-Space Modeling**: The computational impossibility of modeling distributions over million- or billion-dimensional neural network parameter spaces directly, motivating loss-based scalar metrics. - **Per-Sample Distributions vs. Global Thresholds**: Demonstrating that different training examples (e.g., an image of a cat versus an airplane) have vastly different baseline loss distributions, making fixed, global decision thresholds inadequate. - **Evaluation Flaws in Prior Work**: Critiquing the reliance on average-case accuracy across an entire dataset, which obscures whether an attack can reliably identify vulnerable individuals at low false positive rates (FPR).

### Technical Details To estimate the loss distributions, the methodology trains multiple "shadow" models by masking subsets of the training data (typically 50% randomly masked per model). The models are partitioned into two sets for a target image: models that saw the example during training (IN) and models that did not (OUT). Computing the loss on the target image for each set yields two distinct empirical distributions.

While a non-parametric histogram would require training thousands of models (e.g., 10,000 models), the authors fit parametric Gaussian distributions to each set, reducing the requirement to approximately 16–32 models (or as few as 8). The crossing point of these fitted Gaussians provides a calibrated, per-example decision threshold. Furthermore, the likelihood ratio between the distributions provides a calibrated confidence estimate for any given prediction.

Evaluating across ROC curves in log-log scale reveals that prior attacks boasting high average accuracy (e.g., around 59%) perform essentially at random guessing at low false positive rates (<10% FPR). In contrast, the proposed method outperforms prior work both on average accuracy and dramatically at low FPR regimes. The paper also supports both an "online" variant (training models specifically with/without the query point) and an "offline" variant that performs almost as well without retraining per query point.

### Takeaways The speaker highlights a critical conclusion: many existing beliefs in the membership inference literature may need to be re-evaluated. Prior claims that certain defenses (such as weight decay) are ineffective were established only against weak, average-case global threshold attacks, leaving open whether they protect against strong attacks operating at low false positive rates. The primary trade-off of the proposed approach is the computational cost of training multiple models, though its principles also extend downstream to attribute inference and data extraction.