NeFut Logo NeFut
中 Admin Login

[CS.AI] When Reasoning Traces Conflict with Evidence: How Answer Configuration Shapes Evidence Following

Published at: 2026-10-16 22:00 Last updated: 2026-10-18 08:05
#AI #Machine Learning #LLM

A reasoning trace retained at answer time plays two roles: it can boost accuracy and it can clash with the current evidence when they disagree. We disentangle these roles by crossing verified evidence versions with traces generated from each, and answering under two configurations: continuation of the model's own thinking and a separate answer call. Across three model families (gpt-oss, Gemma4, Qwen3) on HotpotQA and MuSiQue, the separate call improves evidence‑following by 29.9 to 60.6 points compared to continuation. Removing the trace lowers accuracy by similar amounts under both configurations. Yet in conflicts, continuation tends to repeat the stale conclusion, while the separate call follows the latest evidence far more often. Thus benefiting from a trace and deferring to it are distinct properties. Further analysis shows that the final line of a trace carries most of its value, gold evidence is most helpful before the trace is written, distractors weaken correction under a separate call, and renewed reasoning—unlike a verification instruction—follows the current evidence in most conflicts. Because trace deletion alone cannot reveal which source an answer follows, evaluations should report both properties together with the answer configuration.

Review

Original Source: https://arxiv.org/abs/2610.20805

[h] Back to Home