Large language model (LLM) agents are increasingly used for long‑horizon tasks, yet current evaluation methods suffer three major problems: terminal‑only scoring ignores intermediate steps and makes error localization costly; single‑reference matching penalizes valid alternative solution paths; and post‑hoc trajectory judging is expensive and cannot stop failing runs early. DynSTEER addresses these issues with a dynamic stage‑wise trajectory evaluation framework. It partitions a rollout into stages anchored at key completed actions, allowing milestone‑level assessment with sufficient context and enabling targeted strategy adjustments. A path‑tolerant milestone graph is compiled from public task views, preserving diverse legitimate strategies without leaking ground truth. Evaluation queries are adaptively routed across multi‑tier judges, and unrecoverable executions are halted online to curb resource waste. Experiments show DynSTEER improves evaluation discriminability by 85.2% over native methods, separates all model pairs with statistical significance, and saves 34.51% of execution steps on failed rollouts.
Review