Spontaneous facial expression database for academic emotion inference in online learning

作     者：Bian, Cunling Zhang, Ya Yang, Fei Bi, Wei Lu, Weigang 

作者机构：Ocean Univ China Dept Educ Technol Qingdao Shandong Peoples R China Shandong Univ Sch Comp Sci & Technol Jinan Shandong Peoples R China Shandong Univ Sch Mech Elect & Informat Engn Weihai Peoples R China SeeleTech Corp San Francisco CA USA Zsbatech Corp Beijing Peoples R China Ocean Univ China Dept Comp Sci & Technol Qingdao Shandong Peoples R China 

出 版 物：《IET COMPUTER VISION》 (IET电脑视觉)

年 卷 期：2019年第13卷第3期

页      面：329-337页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：MOE (Ministry of Education in China) Project of Humanities and Social Sciences [16YJC880057] Natural Science Foundation of Shandong Province [ZR2017MF051] Natural Science Foundation of China 

主　　题：learning (artificial intelligence) neural nets visual databases face recognition emotion recognition inference mechanisms computer aided instruction image classification coefficient external coders image database convolutional neural network-based algorithm data augmentation algorithms adaptive data augmentation algorithm inference performance spontaneous facial expression database online learning learning effect students automatic academic emotion inference facial expressions inference algorithm common academic emotions video clip database 1274 video clips 

摘      要：Academic emotions can produce a great impact on the learning effect. Normally, emotions are expressed externally in the students facial expressions, speech and behaviour. In this paper, the focus is on automatic academic emotion inference based on facial expressions in online learning. Considering the lack of training samples for the inference algorithm, a spontaneous facial expression database is established. It includes the facial expressions of five common academic emotions and consists of two subsets: a video clip database and an image database. A total of 1,274 video clips and 30,184 images from 82 students are included in the database. The samples are labelled by both the participants and external coders. An extensive analysis is carried out on the image database using a convolutional neural network (CNN)-based algorithm to infer self-annotation. Some data augmentation algorithms are applied to improve the algorithm performance. Additionally, an adaptive data augmentation algorithm based on spatial transformer network is introduced, which can remove some confounding factors in the original images. The algorithm can obviously improve the inference performance, which has been proven by comparing some evaluation indicators before and after adoption. Such a database will certainly accelerate the application of affective computing in the educational field.