Credit‑risk models are often trained on proxy labels and deployed under temporal and segment changes. Existing transfer metrics cannot simultaneously separate base‑rate shift, probability‑scale shift, and changes in the feature‑label relationship.
We introduce a locked, multi‑signal audit protocol to detect supervision drift in proxy‑labeled credit‑risk prediction. The protocol consists of five layers: transfer performance assessment, an oracle‑gap probe, a calibration diagnostic, feature‑label stability detection, and a synthetic positive control. All thresholds and decision rules are fixed before interpretation, yielding a bounded reading as the designed outcome.
The study uses the public LendingClub dataset covering 2013‑2016 and evaluates cross‑segment transfer. Results show stable ranking and small oracle gaps.
The clearest temporal signal is a mismatch between prevalence and probability scale; an intercept‑only calibration diagnostic substantially reduces this error, though its root cause cannot be identified from the released data.
The positive‑control experiment responds only to larger injected shifts; subtler drift cannot be ruled out.
Mapping diagnostic patterns to governance actions provides conceptual guidance, but these mappings are not empirically validated in this work.
Review