Machine learning models often achieve high test accuracy while relying on demographic or acquisition shortcuts. We introduce counterfactual (CF) marginalisation as a test‑time evaluation procedure to assess classification robustness against such nuisance variables. The method assumes a CF image generator; for each test image we intervene on parent nuisance variables (e.g., age, sex) to produce CF variants, then average the model predictions over a target intervention distribution. This yields predictions that marginalise demographic effects yet preserve patient‑specific latent information. Using these predictions we define metrics for CF risk, calibration, stability and worst‑case sensitivity. Experiments on several datasets demonstrate the framework’s usefulness for quantitative robustness evaluation.
Review