Experience-based learning enables large language model (LLM) agents to improve by reusing knowledge from past executions. Existing approaches typically rely only on final task outcomes as feedback, overlooking the execution process and limiting the ability to evaluate and optimize experience based on trajectory quality. To overcome this limitation, we introduce the TQG-Exp framework, which incorporates process feedback. The key component is Trajectory Quality Loss (TQL), which jointly considers task completion and process quality, providing a learning signal beyond final outcomes. Using this signal, we perform paired rollouts with and without retrieved experience, comparing the quality differences to guide experience optimization. Representative trajectories are selected based on outcomes and trajectory quality to extract task‑level and task‑type‑level experiences. The quality gap between paired groups estimates the contribution of retrieved experience and updates its utility. The learned utility is then combined with semantic relevance to steer subsequent experience retrieval. Experiments on ALFWorld, WebShop, and ScienceWorld with DeepSeek‑V4‑Flash and Qwen3.5‑Flash backbones show that TQG‑Exp improves Avg.@3 by 4.50 and 8.55 percentage points respectively, outperforming the strongest baselines.
Review