An Artificial Intelligence-Based Multiscale EEG Signal Decoding Framework For Enhanced Cross-Subject Emotion Recognition

Abstract

Emotion recognition using electroencephalography (EEG) is a key challenge in affective/brain-computer interface (Brain-Computer Interaction) design but has poor subject-to-subject transferability. Many aspects, such as inter-individual variability in emotional expression, electrode impedance, skull conductivity, and anatomical structure, result in classifiers being trained on one set of subjects performing poorly when applied to an unseen individual. In this study, we propose an Artificial Intelligence-based framework named Multiscale Domain-Adversarial Fusion Network (MS-DAFN), which increases the number of branches of a multiscale convolutional encoder that extracts fine-, mid-, and coarse-scale holistic spectral-spatial representation from raw EEG signals; models inter-channel and temporal dependencies in raw EEG signals using a graph convolutional and bidirectional long short-term memory (Bi-LSTM) module; fuses the extracted descriptors using channel-attention and temporal-attention; and aligns the feature distributions of source and target subjects by employing an adversarial gradient-reversal branch. It was tested on a leave-one-subject-out (LOSO) cross-subject protocol on the DEAP and SEED benchmark datasets and compared with the support vector machine (SVM), convolutional neural network (CNN), bidirectional long short-term memory network (BiLSTM), and Transformer encoder (TA). The proposed model not only had the best mean CS accuracy and F1-score across all three datasets, but also ablation experiments showed that there were complementary and statistically distinguishable improvements from the multiscale decomposition, graph-temporal encoder, and domain-adversarial alignment branch. These results suggest that a reasonable approach towards emotion-recognition systems that perform well across different subjects without per-subject calibration is provided by multi-scale spectral-spatial decoding combined with adversarial subject-invariant learning.