Large language models (LLMs) achieve strong results on many NLP tasks, yet their ability to perform multi‑step logical reasoning remains insufficiently verified. Chain‑of‑Thought (CoT) prompting improves reasoning by asking models to generate intermediate steps, but it lacks a mechanism to assess the coherence between those steps. This paper introduces a lightweight evaluation strategy that leverages query‑key (Q‑K) alignment within transformer attention heads to measure logical consistency. By performing a single forward pass on a carefully selected set of heads and extracting a “QK‑score”, we obtain latent representations that reliably separate valid from invalid inferences, offering a more scalable alternative to traditional ablation‑based methods. Empirical results on several logical reasoning benchmarks show that the proposed metric is more robust to distractors and captures deeper reasoning chains. Experiments span models ranging from 1.5 B to 70 B parameters.
Overall, the QK‑score provides an intuitive and efficient measure of logical consistency, promising valuable applications in future model evaluation and fine‑tuning. Review