NeFut Logo NeFut
中 Admin Login

[CS.AI] RITE: Evaluating End-to-End Resilience to Silent Tool Errors in E-Commerce Agentic Systems

Published at: 2026-10-22 22:00 Last updated: 2026-10-25 13:04
#AI #Machine Learning #LLM

Tool‑using LLM agents often answer e‑commerce queries by chaining multiple tool calls. The expected output of a tool can differ between development and execution, allowing a plausible yet wrong value to be returned without an explicit failure signal. Such silent errors may propagate through later decisions and corrupt the final answer.

To measure an agent's resilience across the whole pipeline, we define the Silent‑error Resilience Task. The task asks whether the agent can still fulfill the original query after an intermediate state has been silently corrupted.

We build the RITE benchmark from human‑written industrial workflows and production logs. Errors are injected into fixed workflow prefixes according to a semantic error taxonomy, the recorded tool behavior is then frozen, and agents are allowed to continue freely. The benchmark spans nine e‑commerce workflows, 167 matched units, and yields 501 instances of silent errors, correct values, and explicit failures.

Experiments on ten recent LLMs show that silent errors cut task success roughly in half and are consistently harder than explicit failures. The successive bottlenecks are error recognition, use of an available verification path, and integration of the resulting evidence into later reasoning.

To test whether resilience can be improved, we propose a lightweight State Checker intervention that inspects state consistency after an error injection and offers corrective hints.

Review: RITE provides a systematic way to evaluate and enhance agents' fault tolerance to hidden tool errors in realistic e‑commerce settings, highlighting multi‑stage challenges from detection to evidence utilization and demonstrating the promise of simple state‑checking mechanisms.

Original Source: https://arxiv.org/abs/2610.27777

[h] Back to Home