Hierarchical CNN-based real-time fatigue detection system by visual-based technologies using MSP model

作     者：Gu, Wang Huan Zhu, Yu Chen, Xu Dong He, Lin Fei Zheng, Bing Bing 

作者机构：East China Univ Sci & Technol Signal & Informat Proc Shanghai Peoples R China 

出 版 物：《IET IMAGE PROCESSING》 (IET影像处理)

年 卷 期：2018年第12卷第12期

页      面：2319-2329页

核心收录：

学科分类：0808[工学-电气工程] 1002[医学-临床医学] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Shanghai Science and Technology Committee [17DZ1100808  17DZ1100803] 

主　　题：face recognition learning (artificial intelligence) image resolution object detection computer vision neural nets cameras feature extraction pre-trained network - multitask CNN face detection eye mouth regions mouth state detection MSP-Net multiresolution input images open mouth parameters real-time system hierarchical CNN-based real-time fatigue detection system visual-based technologies MSP model driver multitask hierarchical CNN scheme convolutional neural network model multiscale pooling 

摘      要：Visual-based technologies are very useful and meaningful to driver s fatigue detection. In this study, the authors present a multi-task hierarchical CNN scheme for fatigue detection system and propose a convolutional neural network (CNN) model with multi-scale pooling (MSP-Net). Multi-task includes three tasks: face detection, eye and mouth state detection and fatigue detection. First, they use a pre-trained network - multi-task CNN for face detection extracting eye and mouth regions. Then, the main work of this study, eye and mouth state detection is processed by MSP-Net, which can fit multi-resolution input images captured from variant cameras excellently. For the third step, the percentage of eyelid closure over the pupil over time (PERCLOS) parameters and the frequency of open mouth (FOM) parameters are used to detect fatigue, and the FOM parameters are proposed by ourselves. Besides, they successfully port the system to the embedded platform (the NVIDIA JETSON TX2 development board) and test on real driving scene. The results show that their system performs well and is robust to complex environments and is in line with the demand of real-time system.