This work employs SCRIBE, an interpretable contextual semantic representation spanning 65 neuro‑cognitively grounded properties, to dissect the alignment between the large language model LLaMA2-7B and brain activity. Using naturalistic fMRI recordings from seven participants listening to the Moth Radio Hour narratives, we performed variance partitioning between LLaMA2-7B predictions and SCRIBE embeddings, separating the brain‑predictive variance shared by both from the variance unique to each.
Across the whole brain and language‑related regions, shared variance dominates, yet both representations retain significant unique contributions. We then examined whether the concreteness of individual semantic properties relates to shared versus SCRIBE‑unique variance. Concrete properties exhibit greater shared variance with LLaMA2-7B throughout the brain and most regions of interest, whereas abstract properties do not show increased SCRIBE‑unique variance.
Finally, property ablation analyses reveal that semantic contributions span multiple dimensions, producing cumulative effects on both shared and SCRIBE‑unique variance. In sum, for LLaMA2-7B on this dataset, a substantial portion of brain‑LLM alignment is tied to interpretable semantic information, while also uncovering complementary brain‑predictive semantics beyond the model’s own representation.
Review: The study demonstrates a feasible pipeline for linking neuro‑cognitive semantic frameworks with large language models, offering fine‑grained insight into brain‑LLM alignment and highlighting the role of semantic dimensions not captured by the model itself.