Brain‑computer interfaces decode neural activity, yet conventional class‑label supervision (CLS) leaves inter‑class relationships implicit. We introduce Language as Supervision (LaS), which aligns neural features with natural‑language class descriptions via learned projections and predicts classes using candidate text prototypes.
Seven encoder architectures were evaluated on visual, motor, and speech decoding tasks using both scalp and intracranial recordings. LaS improves speech‑decoding accuracy by up to $17.15\text{pp}$ over matched CLS baselines. Its representations enable reuse of candidate sets and support label‑supervised subject and session adaptation. Moreover, LaS allows conditional unseen‑frequency decoding in steady‑state visual evoked potential (SSVEP) responses, provided the class‑description correspondence is correct. Dual‑sided attribution visualizations reveal the contributions of neural signals and textual descriptions to the same matching score.
These findings suggest that textual descriptions can serve as a general‑purpose reference for learning and interpreting neural decoders.
Review