Micro lip reading, is characterized by tiny lip movements when someone is speaking. It is highly preferred in robotics and automation, such as a social robot for geriatric care, a patrol robot system for hospitals, an...
详细信息
Micro lip reading, is characterized by tiny lip movements when someone is speaking. It is highly preferred in robotics and automation, such as a social robot for geriatric care, a patrol robot system for hospitals, and speech system for hearing impaired individuals, etc. However, it has not been well studied before. In this paper, we shed the light to this research topic. A labelled micro lip reading dataset (i.e. HUST-LMLR) of 399 video samples is first established. The samples are captured from the unconstrained movies. One key challenge for micro lip reading in the wild lies on extracting fine features of lip movements effectively and robustly. In this paper, we pay the research efforts to address this issue from two aspects: facial context attention and feature extraction. First, we propose a multi-task learning model for micro lip reading in the wild for the first time. It concerns micro lip reading and facial landmark detection jointly for capturing global face context with soft attention, to facilitate micro lip reading. Second, we propose that motion feature should be concerned and combined with appearance feature jointly to characterize tiny lip movements effectively. Finally, the experiments on HUST-LMLR demonstrate the challenges of our dataset and our proposed approach results in a remarkable improvement of the state-of-the-art by over 26% WER in HUST-LMLR. Furthermore, results in a well-known public dataset LRS2 also show the generalization and superiority of our approach. If accepted, we will publish our HUST-LMLR dataset and relative supporting matierials at https://***/Micro-lip-reading/.
Exercise monitoring systems for rehabilitation are usually not able to pinpoint the exact part for patients' exercise. The research objective is to develop the projection-based motion recognition (PMR) algorithm b...
详细信息
Exercise monitoring systems for rehabilitation are usually not able to pinpoint the exact part for patients' exercise. The research objective is to develop the projection-based motion recognition (PMR) algorithm based on depth data and wide-accepted methods to solve this matter. We regard a motion trajectory as a combination of basic posture units, and then project the basic posture units onto a 2-D space via a projection mapping. Each motion trajectory is transformed to a 2-D motion trajectory map by sequentially connecting the basic posture units involved in the motion trajectory. Finally, we employ a convolutional neural network (CNN)-based classifier to classify the trajectory maps. Accurate classification rate reaches as high as 95.21%. The originality of PMR algorithm lies in (1) it has the generalization capability to some extent since it only adopts popular methods and contains an essential and comprehensive mechanism;(2) the resultant trajectory map may reveal the information about how well a patient execute the rehabilitation assignments.
Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing. Despite previous efforts, most of existing focus on addressing the ey...
详细信息
Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing. Despite previous efforts, most of existing focus on addressing the eyeblink detection problem under constrained indoor conditions with relative consistent subject and environment setup. Nevertheless, towards practical applications, eyeblink detection in the wild is highly preferred, and of greater challenges. In this paper, we shed the light to this research topic. A labelled eyeblink in the wild dataset (i.e., HUST-LEBW) of 673 eyeblink video samples (i.e., 381 positives, and 292 negatives) is first established. These samples are captured from the unconstrained movies, with the dramatic variation on face attribute, head pose, illumination condition, imaging configuration, etc. Then, we formulate eyeblink detection task as a binary spatial-temporal pattern recognition problem. After locating and tracking human eyes using SeetaFace engine and KCF (Kernelized Correlation Filters) tracker respectively, a modified LSTM model able to capture the multi-scale temporal information is proposed to verify eyeblink. A feature extraction approach that reveals the appearance and motion characteristics simultaneously is also proposed. The experiments on HUST-LEBW reveal the superiority and efficiency of our approach. The comparisons with the existing state-of-the-art methods validate the advantages of our manner for eyeblink detection in the wild.
Road friction coefficient, as the most critical variable that controls vehicle motion, has significant impact on the optimal motion control and the warning of slippery road. However, due to the complexity of tire char...
详细信息
Road friction coefficient, as the most critical variable that controls vehicle motion, has significant impact on the optimal motion control and the warning of slippery road. However, due to the complexity of tire characteristics and vehicle dynam- ics models, estimating road friction coefficient with acceptable accuracy in various scenarios is still an unsolved research question in the field of vehicle dynamics. Currently, most of road friction estimation algorithms are built based on the vehi- cle dynamics or acoustic effect from tire, so carefully-tuned model works only on the limited scenarios and is not generalized well for different conditions. Besides, the current state-of-the-art algorithm still experiences the low confidence of esti- mation when the tire is not excited to an adequate level. In this thesis, we build more generalized models using machine learning methods: applying Echo State Net- works ESNs to build on-board road friction estimation algorithm; proposing Hidden Markov Model-based clustering framework to model the spatial-temporalpattern of the road friction over different geometrical locations. We obtain substantial accu- racy improvement of estimation algorithm compared to the in-house physical-based estimation algorithm. And we are able to extract the underlying spatial-temporalpatterns of road friction by the proposed method, which enables to model the statis- tics of reality for simulation as well.
In this paper, we propose a self-organizing feature map-based (SOM) monitoring system which is able to evaluate whether the physiotherapeutic exercise performed by a patient matches the corresponding assigned exercise...
详细信息
In this paper, we propose a self-organizing feature map-based (SOM) monitoring system which is able to evaluate whether the physiotherapeutic exercise performed by a patient matches the corresponding assigned exercise. It allows patients to be able to perform their physiotherapeutic exercises on their own, but their progress during exercises can be monitored. The performance of the proposed the SOM-based monitoring system is tested on a database consisting of 12 different types of physiotherapeutic exercises. An average 98.8% correct rate was achieved.
Early detection of bio-terrorist attacks is an important problem in public health surveillance. In this paper, We focus on the detection and characterization of outdoor aerosol releases of Bacillus anthracis. Recent r...
详细信息
ISBN:
(纸本)9783540897453
Early detection of bio-terrorist attacks is an important problem in public health surveillance. In this paper, We focus on the detection and characterization of outdoor aerosol releases of Bacillus anthracis. Recent research has shown promising results of early detection using Bayesian inference from syndromic data in conjunction with meteorological and geographical data [1]. Here we propose an extension of this algorithm that models multiple days of syndromic data to better exploit the temporal characteristics of anthrax outbreaks. Motivations, mechanism and evaluation of our proposed algorithm are described and discussed. An improvement is shown in timeliness of detection on simulated outdoor aerosol Bacillus anthracis releases.
暂无评论