Multi‑teacher on‑policy distillation (MOPD) extends conventional on‑policy distillation by routing domain‑specific data to specialized teachers and providing dense token‑level supervision on student‑generated trajectories.\ \ The routing only separates supervision sources; gradients from all domains still update a shared set of student parameters. When teachers differ greatly in response style and domain expertise, supervision that works in isolation can cause persistent gradient conflicts during joint optimization, leading to capability degradation.\ \ To tackle this issue, we introduce Cross‑Teacher Guarding (Road MOPD). The key idea is to add non‑routed teachers as auxiliary cross‑domain critics that evaluate the same student trajectories. Coordination is achieved through two mechanisms:\ \
- Cross‑teacher‑guided credit assignment – teacher agreement is used to amplify token‑level corrections supported by multiple teachers while attenuating conflicting signals.\
- Preventive multi‑teacher gradient projection – auxiliary gradients computed on the same trajectories constrain potentially conflicting updates by projecting them onto a subspace that does not interfere with target‑domain learning signals; the projection strength is adaptively adjusted based on the degree of conflict.\ \ Empirical studies show that Road MOPD mitigates degradation in the disadvantaged domain, achieving over a 5‑point gain under high‑conflict conditions, and improves overall performance in low‑conflict and three‑domain settings.\ \ Review: By explicitly modeling teacher agreement and conflict, cross‑teacher guarding offers a general framework that balances multi‑domain learning with shared parameters, making multi‑teacher distillation more robust for real‑world deployment.