Rubric‑based reinforcement learning supplies interpretable, fine‑grained supervision for aligning large language models (LLMs) on open‑ended generation tasks. Static rubric criteria, however, quickly become saturated or lie beyond the policy’s exploration frontier, making reward signals insufficiently discriminative. Recent works that steer sampling or evolve rubrics do not explicitly tie rubric difficulty to the evolving policy’s capabilities, leading to a persistent mismatch between the optimization objective and the current capability frontier.\
We introduce CoCA (Co‑Evolving Policy and Rubric via Capability Awareness), a closed‑loop framework that jointly adapts the rubric and the policy through criterion‑level diagnosis. CoCA contrasts query‑only rollouts with rubric‑guided rollouts to assess each criterion relative to the current policy, recalibrates criteria whose difficulty is misaligned, and pinpoints capability gaps revealed by the guidance for frontier‑based learning. Leveraging this insight, CoCA either expands rubric‑conditioned frontiers via guided optimization or transfers qualified gains through on‑policy self‑distillation. As the policy updates, diagnostic accuracy improves and distillation becomes more reliable, completing the co‑evolution loop.\
Extensive experiments across diverse benchmarks and LLM scales demonstrate that CoCA consistently outperforms existing rubric‑based RL methods. Moreover, CoCA promotes the internalization of guidance‑revealed capabilities into query‑only behavior while mitigating capability degradation.\
Review