Existing Mixture‑of‑Agents (MoA) setups treat query routing and agent fine‑tuning as separate stages, which prevents routing strategies from adapting to agents that improve after training and blocks agents from achieving data‑driven specialization. To address this gap, we introduce CERA‑MoA (Co‑Evolving Router with continually learning Agents for Mixture‑of‑Agents), an iterative reinforcement‑learning framework where a dynamic router and independent agent policies evolve together.
A predictive familiarity estimator is built on mid‑layer hidden states to gauge semantic competence among agents, avoiding the cost of full rollouts. Using these familiarity scores, a cumulative‑threshold adaptive routing mechanism dynamically activates a tailored minimal subset of agents, balancing task performance with efficiency.
During training, the system proactively assigns targeted samples to agents whose competence is currently highest, encouraging differentiation of capabilities across the agent pool.
Extensive experiments across diverse domains show that CERA‑MoA consistently outperforms state‑of‑the‑art static‑agent routing and fixed‑workflow fine‑tuning baselines in both success metrics and resource usage.
Review: CERA‑MoA’s co‑evolution of router and agents enables adaptive routing and continual agent specialization, offering a compelling new direction for large‑model ensemble systems.