Abstract
Decoding continuous three-dimensional (3D) motor imagery (MI) using non-invasive electroencephalography (EEG)-based brain-computer interfaces (BCIs) remains challenging due to signal variability and residual decoding errors. Deep learning architectures such as convolutional neural network-long short-term memory (CNN-LSTM) models can capture spatial and temporal dynamics for continuous kinematic decoding; however, systematic residual errors persist in predicted trajectories. We propose a two-stage decoding framework that applies reinforcement learning (RL) to perform residual kinematic correction on the outputs of a CNN-LSTM decoder (CNN-LSTM-RL). The RL agent is trained offline without direct EEG input and instead operates on predicted kinematic trajectories to optimize movement accuracy relative to target trajectories.
Decoding performance was quantified using Pearson correlation coefficients ($r$) and Root Mean Square Errors (RMSE) along the $x, y$, and $z$ axes. Compared to CNN-LSTM applied alone, CNN-LSTM-RL improved the mean correlation from $0.5076$ to $0.7181$ ($p = 0.0005$) in 2D and from $0.6420$ to $0.7780$ ($p = 0.0059$) in VR, with relative gains of $41.5\\%$ and $21.2\\%$, respectively. Correspondingly, RMSE was reduced from $0.0890$ to $0.0532$ (2D, $p < 0.0001$).
Blogger's Review: This study significantly enhances the accuracy of motor decoding in brain-computer interfaces through reinforcement learning, showcasing the immense potential of combining deep learning with RL. It not only advances neural decoding technology but also provides new insights and methods for future BCI applications. Notably, the introduction of RL allows the system to optimize in an unsupervised manner, greatly enhancing the model's flexibility and adaptability.