In long-horizon graphical-user-interface tasks, deciding whether a trajectory fulfills its instruction is crucial for evaluating and training computer-use agents. This judgment has long relied on rule-based evaluation, which struggles to align with human intention and becomes obsolete with app updates or content drift. Existing model-based judges attempt to address these issues but still leave a performance gap compared to rule-based evaluation. We propose the SeekJudge framework, where four role-specialized agents—Condense, Ground, Seek, and Analyze—reach a verdict through a Seek-Analyze loop over the trajectory. A seed-calibrated distillation pipeline trains a specialized $9$B model to serve as a shared backbone for all four agents. Measured by downstream success rates on held-out RL test goals, SeekJudge is the first practical model-based reward to match or surpass native rule-based supervision in online RL. Beyond accuracy, SeekJudge provides step-level judgments, runs far cheaper than a closed-source large model, and maintains a small per-call context that scales to much longer trajectories. Additionally, we contribute a general architectural improvement to the reward server that speeds up judging in RL. Together, these advancements make model-based rewards a practical drop-in for rule-based supervision in CUA reinforcement learning.
Blogger's Review: The SeekJudge framework significantly enhances the practicality of model-based reward systems by introducing multiple specialized agents, addressing the limitations of traditional methods. This innovation not only improves evaluation accuracy but also reduces costs, showcasing a major advancement in the reinforcement learning field. Its architectural improvements also provide valuable insights for future research.