NeFut Logo NeFut
中 Admin Login

[CS.AI] When Reasoning Looks Like Hidden Understanding: How Chain-of-Thought Fosters Anthropomorphism, Perceived Epistemic Superiority, and Cognitive Deference

Published at: 2026-11-11 22:00 Last updated: 2026-11-12 00:51
#AI #Machine Learning #LLM

As generative models become pervasive in products, a belief system about their capabilities emerges. A common misconception equates the form of language with “understanding,” yet models merely reproduce statistical patterns without grasping meaning. A further error is to assume models can “explain their reasoning,” especially when Chain‑of‑Thought (CoT) is taken as a transparent explanation. The paper argues that this misreading of form as meaning leads users to defer cognitively to AI, treating the system as an intentional “mind” and respecting its perceived epistemic authority. Consequently, users commit the “fallacy of composition,” generalising narrow task success to broader competence. Using medicine as a case study, the authors illustrate the misleading risks of CoT. They conclude that, although CoT can improve output accuracy and moderation, exposing model reasoning to end‑users should be actively discouraged, and AI literacy training must explicitly warn of its hazards. Review

Original Source: https://arxiv.org/abs/2611.13456

[h] Back to Home