SWE-Critix introduces a post‑training pipeline for training generalist verifiers that can assess coding‑agent execution trajectories.
Coding agents are increasingly used for end‑to‑end software engineering, yet a ready‑made oracle is often missing. Their trajectories may contain hundreds of interactions with code repositories and external tools, making verification difficult.
Existing verifiers either rely on generic LLMs without specialization or train dedicated critics that only output scalar rewards, lacking fine‑grained explanations.
Given a task and its trajectory, SWE‑Critix predicts a completion label and generates a natural‑language analysis that explains the judgment. The pipeline first applies supervised fine‑tuning (SFT) and then reinforcement learning (RL) to improve performance.
We train a series of 30B‑parameter models on trajectories from CodeForge‑Preview. On a held‑out test set, the base Qwen3‑30B‑A3B reasoning model achieves 57% accuracy, which rises to 66% after SFT and reaches 78% after RL.
Our models, datasets, and training code are released to support further research.
Review