This audit examines the reliability of unlearning decisions by analyzing 263 released batch‑normalized vision model checkpoints. The core of the audit compares the verdict numbers published by an unlearned model and its retrained reference, together with their batch‑normalization statistics—statistics that were never written by a gradient step nor recorded in the release logs.
Researchers refit these statistics on the kept data while keeping the weights bit‑identical. Among 221 checkpoints, 47 move beyond the spread shown by their own release seeds, indicating that the movement originates from the checkpoint’s property rather than from a shift in the method’s average.
A further experiment shows that swapping kept records for removed ones changes an individual batch‑norm cell by almost nothing, whereas the drift of a checkpoint’s shipped state from any refit reliably tracks the observed movement.
The practical impact appears at the verdict level: out of 263 checkpoints, twelve verdicts cross the decision threshold, four are cleared within a measured recalibration budget, two are cleared in every replicate, and a population we trained near its own criterion yields none.
Consequently, releases should name the batch‑normalization fitting convention alongside the reported number, especially for batch‑normalized vision models where this channel exists.
Review