Audit Finds Batch-Normalization Stats Skew Machine Unlearning Evaluations

A new audit examines 263 publicly released checkpoints that use batch normalization and finds that reported unlearning results shift depending on which version of those statistics is used. Because batch-norm statistics are not produced by gradient updates and are rarely documented in model releases, refitting them on retained data can change the numbers an evaluation relies on. The authors argue that this makes some unlearning verdicts unreliable, since apparent forgetting may reflect checkpoint bookkeeping rather than the removed data genuinely being gone.