A Novel Domain Adversarial Networks Based on 3D-LSTM and Local Domain Discriminator for Hearing-Impaired Emotion Recognition

作     者：Tian, Zekun Li, Dahua Yang, Yi Hou, Fazheng Yang, Zhiyi Song, Yu Gao, Qiang 

作者机构：Tianjin Univ Technol Sch Elect Engn & Automat Tianjin Key Lab Control Theory & Applicat Complica Tianjin 300384 Peoples R China Tianjin Univ Technol Maritime Coll Tianjin Key Lab Control Theory & Applicat Complica Tianjin 300384 Peoples R China 

出 版 物：《IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS》 (IEEE J. Biomedical Health Informat.)

年 卷 期：2023年第27卷第1期

页      面：363-373页

核心收录：

学科分类：0710[理学-生物学] 0808[工学-电气工程] 1001[医学-基础医学(可授医学、理学学位)] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China National Natural Science Foundation of China 

主　　题：Electroencephalogram (EEG) emotion recognition hearing-impaired subjects domain adaptation neural network (DANN) 

摘      要：Recent research on emotion recognition suggests that deep network-based adversarial learning has an ability to solve the cross-subject problem of emotion recognition. This study constructed a hearing-impaired electroencephalography (EEG) emotion dataset containing three emotions (positive, neutral, and negative) in 15 subjects. The emotional domain adversarial neural network (EDANN) was carried out to identify hearing-impaired subjects emotions by learning hidden emotion information between the labeled data and the data with no-label. For the input data, we propose a spatial filter matrix to reduce the overfitting of the training data. A feature extraction network 3DLSTM-ConvNET was used to extract comprehensive emotional information from the time, frequency, and spatial dimensions. Moreover, emotion local domain discriminator and emotion film group local domain discriminator were added to reduce the distribution distance between the same kinds of emotions and different film groups, respectively. According to the experimental results, the average accuracy of subject-dependent is 0.984 (STD: 0.011), and that of subject-independent is 0.679 (STD: 0.140). In addition, by analyzing the discrimination characteristics, we found that the brain regions with emotional recognition in the hearing-impaired are distributed in the wider areas of the parietal and occipital lobes, which may be caused by visual processing.