NeFut Logo NeFut
中 Admin Login

[CS.AI] Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

Published at: 2026-10-12 22:00 Last updated: 2026-10-13 00:57
#AI #Machine Learning #Open Source

Open coding agents are typically trained by supervised fine‑tuning on interaction trajectories whose final patches pass a benchmark test suite. Relying solely on this outcome‑level filter ignores the quality of the process: even successful trajectories may contain inefficient patterns such as repetitive file inspection, unstable tool usage, or delayed modifications, which become part of the imitation target.\ \ We introduce TQS (Trajectory Quality Score), a structural scoring framework that operates directly on trajectory structure. TQS requires no LLM judge and runs in linear time. It characterizes trajectory quality through three structural signals organized around structured convergence: information focus, intent‑action alignment, and policy dynamics.\ \ Using 155,144 successful trajectories from a public open‑license corpus, we rank trajectories by TQS and fine‑tune coding agents on filtered subsets. Models trained on top‑ranked trajectories consistently outperform those trained on random or bottom‑ranked subsets on SWE‑Bench‑Verified. Our results indicate that process‑level trajectory quality is an important factor in coding‑agent supervision, and that structural trajectory filtering offers an effective, scalable alternative to LLM‑based data selection.\ \ Review

Original Source: https://arxiv.org/abs/2610.13152

[h] Back to Home