NeFut Logo NeFut
中 Admin Login

[CS.AI] ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing

Published at: 2026-10-17 22:00 Last updated: 2026-10-18 08:05
#AI #Machine Learning #LLM

ClaimScope is a 72‑task benchmark for auditing mathematical claims. Each task asks the model to provide a verdict, scope, counterexample, and repair, covering optimizer‑state transport, finite‑horizon affine recurrences, and exact $2\times2$ linear algebra. We evaluate Qwen3‑8B with direct generation and four matched two‑pass conditions, all starting from the same draft.\ \ A witness checker evaluates only the model’s submitted state, without consulting reference verdicts or supplying a replacement witness. Its feedback still conveys validity information: a verified refutation marks the claim as false, and rejection reasons can suggest structural fixes.\ \ In this evaluation the verifier feedback yields 23/39 accepted counterexamples on invalid claims and 25/72 safe completions, compared with 15/39 and 20/72 for generic feedback without a checker. Strict completion remains 17/72; verdict accuracy changes from 47/72 to 46/72. The claim‑grouped safe‑completion difference is $.094$, with a bootstrap 95% interval of $[.000,.198]$. All seven task‑level gains align with horizon/input‑length diagnostics. A post‑hoc schema‑only correction of the direct outputs reaches the same 25 safe tasks without another model call.\ \ These results separate claim classification from executable evidence, locating the observed gains in witness‑structure correction rather than demonstrating broader reasoning improvement.\ \ Review

Original Source: https://arxiv.org/abs/2610.21581

[h] Back to Home