Curriculum learning typically orders training samples from easy to hard. In GRPO, medium‑difficulty samples are attractive for online selection because they are more likely to provide informative group‑relative learning signals, and prioritizing them can improve training efficiency.
Nevertheless, it is unclear whether persistently selecting only medium‑difficulty samples yields the strongest final model, or whether training can benefit from progressing toward harder samples. Moreover, sample difficulty changes as the policy evolves, so fixed difficulty estimates may become misaligned with the current policy.
To address this, we re‑estimate sample difficulty under the current policy and, inspired by the easy‑to‑hard principle of curriculum learning, design and evaluate a staged curriculum that gradually shifts the active difficulty band toward harder samples.
Experiments on the GSM8K benchmark show that persistent medium‑difficulty selection does not achieve the highest observed performance. Across three random seeds, moving to a moderately harder difficulty band improves test accuracy to $0.6591 \pm 0.0077$, compared with $0.6449 \pm 0.0024$ for the fixed‑medium curriculum. Further progression to an even harder band reduces accuracy to $0.6405 \pm 0.0053$.
These findings demonstrate that staged difficulty progression can outperform a static medium‑difficulty curriculum, but its benefit is non‑monotonic. Effective GRPO curricula should progress beyond medium difficulty while avoiding excessive advancement toward overly hard samples.
Review