Multimodal large language models (MLLMs) can reproduce sensitive, copyrighted, or unsafe information from their training data. Effective unlearning must remove the target information while preserving non‑target content within the same response. Existing approaches usually apply a uniform forgetting objective to whole responses and rely on fixed off‑policy supervision, which leads to coarse token‑level control and misalignment with the model's current behavior.\
We propose CUSE (Counterfactual‑guided Unlearning framework via on‑policy Self‑distillation). It works in two stages. First, counterfactual inputs are created by stripping evidence related to the target; the resulting changes in token probabilities reveal which tokens require stronger forgetting. Second, token‑level forgetting and retention targets are distilled on responses sampled from the current policy. A frozen copy of the original model serves as the teacher, augmented with privileged contexts that emphasize forgetting or retention. This design jointly determines what to forget and what to preserve.\
On the MLLMU‑Bench benchmark, CUSE reduces the forget‑set ROUGE‑L by 33.6% while achieving a 1.2% higher retain‑set ROUGE‑L than the strongest baseline.\
Review: CUSE leverages counterfactual guidance to pinpoint forget‑worthy tokens and employs on‑policy self‑distillation for fine‑grained control, markedly improving unlearning performance in multimodal models without sacrificing retained knowledge.