Existing evaluations of multimodal unlearning usually check only whether a model stops emitting the target information, which hides two serious failure modes. First, knowledge suppression often causes the model to hallucinate facts about the erased subject. Second, refusal training merely changes observable behavior without truly deleting the underlying knowledge; the model may refuse open queries yet still select correct answers in multiple‑choice probing. To address these issues we introduce Forget‑Aware Self‑Distillation (FASD), which aims for honest ignorance – the model genuinely loses the target knowledge and learns to acknowledge its absence. FASD splits the objective into two complementary channels. The behavior channel distills appropriate refusals from an ignorance‑prompted copy of the model, masking target tokens in the teacher distribution to avoid inadvertently re‑teaching forgotten concepts. The knowledge channel directly suppresses ground‑truth answer spans, independent of the model’s generated output. Experiments show that FASD successfully unlearns the target knowledge, prevents information leakage in open‑ended tasks, and preserves performance on retained data. The resulting model not only forgets the specified information but also accurately recognizes the limits of its own knowledge.
Review