Learning Spatial-Spectral-Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment

作     者：Zhang, Pengbo Wang, Xue Zhang, Weihang Chen, Junfeng 

作者机构：Tsinghua Univ Dept Precis Instrument State Key Lab Precis Measurement Technol & Instru Beijing 100084 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING》 (IEEE神经系统与康复工程汇刊)

年 卷 期：2019年第27卷第1期

页      面：31-42页

核心收录：

学科分类：0831[工学-生物医学工程（可授工学、理学、医学学位）] 1002[医学-临床医学] 0808[工学-电气工程] 08[工学] 

基　　金：National Natural Science Foundation of China 

主　　题：Mental workload cross-task 3D convolutional neural network recurrent neural network EEG 

摘      要：Mental workload assessment is essential for maintaining human health and preventing accidents. Most research on this issue is limited to a single task. However, cross-task assessment is indispensable for extending a pre trained model to new workload conditions. Because brain dynamics are complex across different tasks, it is difficult to propose efficient human-designed features based on prior knowledge. Therefore, this paper proposes a concatenated structure of deep recurrent and 3D convolutional neural networks (R3DCNNs) to learn EEG features across different tasks without prior knowledge. First, this paper adds frequency and time dimensions to EEG topographic maps based on a Morlet wavelet transformation. Then, R3DCNN is proposed to simultaneously learn EEG features from the spatial, spectral, and temporal dimensions. The proposed model is validated based on the EEG signals collected from 20 subjects. This paper employs a binary classification of low and high mental workload across spatial n-back and arithmetic tasks. The results show that the R3DCNN achieves an average accuracy of 88.9%, which is a significant increase compared with that of the state-of-the-art methods. In addition, the visualization of the convolutional layers demonstrates that the deep neural network can extract detailed features. These results indicate that R3DCNN is capable of identifying the mental workload levels for cross-task conditions.