NeFut Logo NeFut
Admin Login

[CS.AI] ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations

Published at: 2026-09-03 22:00 Last updated: 2026-09-04 02:14
#algorithm #AI #Machine Learning

Agent evaluations confront two evidentiary questions: can a reported claim be recomputed from retained evidence (sufficiency), and do the retained records cover the committed set of experiments (coverage). Generic logs and hash‑linked transcripts fail to answer either reliably. We introduce ClaimReceipt, a claim‑relative receipt specification and selective verifier that binds typed transaction evidence to a signed experiment manifest and returns PASS, INVALID or INCONCLUSIVE per claim. The specification is frozen before implementation (SHA‑256 18d109...b81). On 1,392 historical buyer‑seller records, a CR‑2 verifier reproduces all five manually labeled audit verdicts, exactly replays 600 deterministic and 792 post‑generation records, makes every one of 13 declared field groups non‑redundant under tested ablations, and returns the expected result on 11/11 semantic faults with zero false positives. We then run a separate prospective CR‑3 epoch: 30 assignments are committed before inference, terminal receipts are signed and chained, and private evidence is encrypted for an auditor. Complete evidence yields coverage and accounting PASS; withholding a single terminal receipt returns INCONCLUSIVE_COVERAGE, while withholding all private openings preserves coverage and protocol verification but leaves economic claims inconclusive, exactly matching a preregistered prediction. Receipt instrumentation adds 0.021% of model‑inference time and 9.9 KB per transaction. A specification‑legibility probe indicates that our frozen specification is not yet unambiguous to an independent reader. Claim verification therefore requires both claim‑sufficient evidence and a committed universe against which omissions become visible.

Review

Original Source: https://arxiv.org/abs/2609.01992

[h] Back to Home