Abstract
Generalizing across subjects remains challenging in invasive neural recordings due to substantial variations in electrode configurations, anatomical structures, and neural signal patterns. To address this inter-subject variability, we propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space and learns a mapping from the aligned neural representations to contextual embeddings.
Specifically, using electrocorticography data collected during natural language comprehension, we estimate the shared space via the shared response model and train a decoder to predict contextual semantic embeddings from projected neural responses. For a held-out subject, we estimate a subject-specific projection into the predefined shared space and directly apply the pretrained decoder without any retraining.
Experimental results demonstrate that the proposed framework consistently outperforms baseline methods across evaluation settings and exhibits a reduced performance drop from source subject to held-out subject testing, indicating improved cross-subject generalization. These results suggest that aligning neural activity into a shared latent space, while decoding in a semantic embedding space, provides an effective strategy for improving cross-subject generalization by reducing subject-specific differences in neural responses while effectively capturing shared stimulus-related representations.
Blogger's Review: The proposed framework significantly enhances cross-subject generalization of neural signals through shared latent space alignment, showcasing promising applications, especially in brain-computer interfaces and neurorehabilitation.