NeFut Logo NeFut
中 Admin Login

[CS.AI] Can LLMs Trust Appropriately? Evaluating Calibrated Reliance on Imperfect Partners

Published at: 2026-10-28 22:00 Last updated: 2026-10-29 00:58
#Machine Learning #LLM #Artificial Intelligence

Large language models (LLMs) are increasingly collaborating with humans or peer AI agents, whose contributions can be incomplete, inconsistent, or outright wrong. Effective collaboration requires calibrated trust: the model must adapt its decisions to rely on, verify, or assist a partner rationally as empirical evidence accumulates.\ We introduce a benchmark comprising 324 scenarios across 23 collaboration categories. Each scenario involves three sequential decisions with explicit transition dynamics, feedback emissions, and cost schedules, allowing exact analytical evaluation of choices against their expected consequences over the remaining interaction.\ Seven state‑of‑the‑art models are evaluated together with 36 controlled‑history probes to separate evidence sensitivity from normatively appropriate action. While the strongest frontier model approaches near‑optimal decisions, other models exhibit recurring pathologies: procuring excessive safeguards, purchasing verification tests with zero expected value of information, prioritizing immediate savings over cooperative learning, and misallocating assistance to suboptimal channels.\ Providing explicit posterior distributions resolves some errors but also introduces new failure modes. The study demonstrates that calibrated trust in autonomous agents critically demands coupling empirical evidence with forward‑looking decision utility.\ \ Review: By combining finely crafted scenarios with rigorous analytical grounding, this work pinpoints fundamental shortcomings in LLMs' collaborative trust and offers clear guidance for future improvements.

Original Source: https://arxiv.org/abs/2610.34107

[h] Back to Home