Under matched experimental conditions, the classifier‑head fine‑tuning (\textsc{cls}) paradigm consistently outperforms the LLM‑as‑Judge sequence‑to‑sequence (\textsc{s2s}) approach on binary hallucination detection. We attribute this gap to the representations induced by the two objectives. Static analysis shows that \textsc{cls} learns a task‑specific discriminative direction, pushing hidden states into a clear two‑cluster geometry with stronger class separation and linearly decodable label information. By contrast, \textsc{s2s} relies on the pretrained language‑modeling head to make decisions, minimizing pressure to reorganize representations, so hidden states remain close to the pretrained backbone. We also find that the \textsc{cls} signal saturates before the final layer, after which upper layers form a statistically interchangeable and largely redundant plateau. Endpoint redundancy alone does not reveal whether these layers are dispensable during training. Training‑dynamics analysis uncovers three observations: 1) a lower pretrained substrate; 2) a narrow upper‑middle \textbf{InfoWindow} where task information emerges; 3) a later‑synchronizing upper plateau that mirrors this signal. Motivated by this mechanism, we remove layers above the InfoWindow. Truncating 31% of the network depth leaves accuracy unchanged within seed noise on Qwen3‑4B and 8B, while reducing latency by about $1.4\times$. The same depth fraction, fixed before any model‑specific analysis, transfers to Llama‑3.2‑3B and Mistral‑7B‑v0.3, and the underlying layer structure reappears on QNLI and SST‑2. We hope this work guides efficient task‑specific LLM design.
Review