Self‑play co‑training where a single language model acts as both coder and test author promises to push code‑generation RL beyond fixed test suites, yet it suffers from two coupled pathologies.
- Permissiveness collapse: the pass‑rate reward is maximised by trivial, non‑discriminative tests, rendering the reward ineffective.
- Concentration bias: i.i.d. sampled tests cluster on modal inputs, inflating estimator variance.
We introduce CoVer (Co‑trained Coder and Verifier), a single‑policy GRPO framework that tackles both failures.
- Information‑gain reward: each self‑generated test is scored by the mutual information between its pass/fail vector $\mathbf{v}$ and a graded ground‑truth correctness signal $y \in [0,1]^m$: $I(\mathbf{v}; y)$. The reward is gated by the sign of their covariance, so only positively discriminative tests receive credit:
$$\text{reward}=I(\mathbf{v}; y)\cdot \mathbf{1}_{\operatorname{cov}(\mathbf{v}, y)>0}$$
- Three‑stage diversity‑aware selection: a candidate pool is pruned through
- Invalidity filtering: discard syntactically broken or runtime‑error tests;
- Input‑string filtering: remove tests that are highly similar in input space;
- Execution‑profile filtering: eliminate behaviorally redundant tests based on execution paths and resource usage.
This step raises the effective sample size of the IG estimator under a fixed execution budget.
Across five benchmarks (LiveBench, MBPP, LiveCodeBench, CodeContests, Code‑Forces), CoVer boosts one‑shot pass@1 by $+5.8$ points at 7B and $+7.1$ points at 14B over the Qwen2.5‑Instruct backbone, achieving the highest macro‑average among all compared methods. Integrated as a drop‑in backbone in the CodeT ranking pipeline, CoVer‑7B adds an extra $+3.5$ points, demonstrating the dual benefit of co‑training for both generation and selection.
Review