Multi-domain fusion deep graph convolution neural network for EEG emotion recognition

作     者：Bi, Jinying Wang, Fei Yan, Xin Ping, Jingyu Wen, Yongzhao 

作者机构：Northeastern Univ Coll Informat Sci & Engn Shenyang 110169 Peoples R China Northeastern Univ Fac Robot Sci & Engn Shenyang 110819 Peoples R China 

出 版 物：《NEURAL COMPUTING & APPLICATIONS》 (神经网络计算与应用)

年 卷 期：2022年第34卷第24期

页      面：22241-22255页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Foundation of National Natural Science Foundation of China [61973065, 52075531] Fundamental Research Funds for the Central Universities of China [N2104008] Innovative Talents Support Program of Liaoning Provincial Universities [LR2020047] Central Government Guides the Local Science And Technology Development Special Fund [2021JH6/10500129] 

主　　题：Emotion recognition Multi-domain fusion features Graph convolution neural network Transfer learning 

摘      要：Electroencephalogram (EEG)-based emotion recognition has become a hot research field, with the most attention given to decoding three basic types of emotional states (i.e., positive, negative, and neutral) by EEG. Traditional EEG emotion recognition is a single feature input mode that cannot cover multiple feature information. For brain function networks, the feature extraction process is very cumbersome. There will also be individual differences in the characteristics of different subjects. Therefore, the theory of graph convolution and brain function connection is introduced into this research, and the multi-domain fusion features input deep graph convolution neural network (MdGCNN) is proposed in this paper. Pearson correlation is used to determine the adjacency matrix. The Sortpooling layer is employed as a bridge between the graph convolution neural layer and the normal neural network layer and sorts the node features in a consistent order. Based on analyzing the characteristics of a single electrode, the brain topology structure features are automatically extracted. Taking MdGCNN as the basic model and considering the method of minimizing the feature distance between the source and the target domain, we propose a transfer learning (TL) emotion recognition model for cross-subject called MdGCNN-TL. Meanwhile, MdGCNN-TL is extended to traverse the target domain of a single subject in a two-to-one domain form. According to the idea of principal component analysis, the transfer model with a high recognition effect is determined with the degree of subject correlation (SC), and MdGCNN-TL is upgraded to MdGCNN-TL-SC. Experimental analysis on the SEED dataset is performed to evaluate the proposed models. Further validation of the model is implemented on the DEAP dataset. The results show that the proposed model has achieved better performance in EEG emotion recognition.