AgentEval is a system that carries a contract‑bound assessment record through tool execution, state and event capture, report qualification, and case maintenance. It cleanly separates what the agent actually did, what the evidence can establish, and whether the conclusion is supported for current use, thus avoiding reliance on mere task completion or hash‑only checks to judge obligation fulfillment.
In experiments across 32 tool‑world configurations, all 30 complete episodes received consistent independent verification, clearly distinguishing satisfied, violated, and unresolved obligations; two interrupted runs were retained in the record. Controlled interventions on 48 evidence views exposed 36 binding faults that hash‑only checks missed, while the agent’s behavior remained unchanged.
Scenario Studio links reviewed cases to 12 two‑turn report sets, with 24 answers independently checked. A separate public benchmark on 192 held‑out executions revealed ten sub‑goals completed despite broader task failures and isolated the impact of missing evidence.
These studies demonstrate that an executable assessment record makes scope, evidential support, and report reuse inspectable throughout an agent evaluation workflow.
Review