NeFut Logo NeFut
Admin Login

[CS.AI] Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning

Published at: 2026-07-22 22:00 Last updated: 2026-07-23 12:33
#AI #Machine Learning #optimization

Abstract

Supervised fine-tuning (SFT) of open-weight LLMs on expert agent trajectories has emerged as a prominent approach to building capable code agents without reliance on proprietary models. A central yet underexplored question is how trajectory quality and quantity jointly shape model performance. We present a systematic empirical study of trajectory data filtering for LoRA fine-tuning of Qwen2.5-Coder-7B-Instruct on the SWE-trajectory dataset (67,074 trajectories, of which 32,161 are resolved).

We propose a two-axis quality scoring framework -- Efficiency and Style -- and evaluate it through 16 controlled experiments spanning strategy, scale, and ablation analyses. Since 7B-scale models attain near-zero SWE-bench resolve rates, we adopt cross-entropy (CE) loss on held-out trajectories as the primary metric, validated via first-action generation: CE loss and ROUGE-L are perfectly rank-correlated (Spearman $\rho$ = -1.00), with limited-sample evidence supporting but not conclusively establishing this proxy.

Our results reveal a scale-dependent quality-quantity trade-off: at small scales, doubling the dataset (500 to 1,000) yields ~12.7% CE-loss reduction whereas the TopQ-Random gap stays 0.10; at 2,000 trajectories this same gap widens to 3.6% (p = 0.016). Ablation further identifies error-retry rate as the dominant sub-dimension, performing comparably to the full composite ($\Delta$).

Blogger's Review: This study delves into how the quality and quantity of trajectory data impact the performance of code agent models. The proposed scoring framework offers crucial insights for future fine-tuning strategies. The systematic experimental design not only reveals the influence of dataset scale but also provides empirical evidence for optimizing data processing, holding significant theoretical and practical implications.

Original Source: https://arxiv.org/abs/2607.17205

[h] Back to Home