Abstract
Tool-using LLM agents are typically evaluated based on final-answer correctness or LLM judges, neither of which captures the production process of an answer. In safety-critical environments, the procedure itself is integral to correctness. This paper introduces AgentLTL, a language derived from First-Order Linear Temporal Logic (FO-LTL) that expresses procedural rules over agent traces. It yields a deterministic, judge-free compliance score.
In this framework, a single specification drives two usages. The first is harnessing: the constraints score completed traces, or gate tool calls by checking each prefix online before execution. The second is finetuning: the score serves as a dense reward. On a benchmark spanning ordering, branching, iteration, and grounding, block-and-warn harnessing improves compliance on five of seven models. Finetuning with the same reward yields +38 and +17.5 percentage point gains in accuracy and compliance on held-out patterns, including unseen tool-name aliases. These findings indicate that the model acquires procedural structure rather than merely memorizing surface tool names and procedures.
Blogger's Review: AgentLTL introduces an innovative approach to assess and enhance procedural compliance in LLM agents, particularly crucial in high-stakes applications. By implementing a judge-free compliance scoring mechanism, it significantly improves model performance in complex tasks, demonstrating the effectiveness of procedural design. This framework offers a fresh perspective for the development and evaluation of tool-using LLM agents in the future.