NeFut Logo NeFut
Admin Login

[CS.AI] Task-Adaptive Rubrics for GUI Reward Modeling

Published at: 2026-08-26 22:00 Last updated: 2026-08-29 12:04
#AI #Machine Learning #optimization

Recent work on GUI agents has shifted toward outcome reward modeling, which assigns rewards by checking whether an executed trajectory satisfies the success criteria implied by a user instruction. Existing GUI reward verifiers often under‑specify how to construct these criteria for each task instance. Whether they rely on generic rubric structures or implicit model reasoning, their judging criteria lack task adaptivity: they may transfer checks across tasks, overlook concrete constraints in the current instruction, or become overly strict by enforcing unstated requirements. To overcome this limitation, we introduce AdaptRubric, a coarse‑to‑fine rubric framework that builds task‑adaptive judging criteria through a category‑level coarse stage and an instance‑level fine stage. AdaptRubric first routes the instruction to a GUI task family and retrieves reusable family‑level criteria, then generates compact cues at the instance level to capture concrete values, scopes, and constraints in the instruction. Across offline reward evaluation and online reinforcement‑learning optimization, AdaptRubric consistently outperforms prior reward agents under a matched image budget, improving F1 by 3.6 points and achieving a 4.23‑point gain in task success.

Blogger's Review: AdaptRubric offers a promising direction for task‑adaptive reward modeling in GUI agents.

Original Source: https://arxiv.org/abs/2608.24174

[h] Back to Home