Developing phoneme-based lip-reading sentences system for silent speech recognition

作     者：Randa El-Bialy Daqing Chen Souheil Fenghour Walid Hussein Perry Xiao Omar HKaram Bo Li 

作者机构：School of EngineeringLondon South Bank UniversityLondonUK Faculty of Informatics and Computer ScienceBritish University in EgyptCairoEgypt School of Electronics and InformaticsNorthwestern Polytechnical UniversityXi'anChina 

出 版 物：《CAAI Transactions on Intelligence Technology》 (智能技术学报（英文）)

年 卷 期：2023年第8卷第1期

页      面：129-138页

核心收录：

学科分类：081203[工学-计算机应用技术] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：deep learning deep neural networks lip-reading phoneme-based lip-reading spatial-temporal convolution,transformers 

摘      要：Lip-reading is a process of interpreting speech by visually analysing lip *** research in this area has shifted from simple word recognition to lip-reading sentences in the *** paper attempts to use phonemes as a classification schema for lip-reading sentences to explore an alternative schema and to enhance system *** classification schemas have been investigated,including characterbased and visemes-based *** visual front-end model of the system consists of a Spatial-Temporal(3D)convolution followed by a 2D *** utilise multi-headed attention for phoneme recognition *** the language model,a Recurrent Neural Network is *** performance of the proposed system has been testified with the BBC Lip Reading Sentences 2(LRS2)benchmark *** with the state-of-the-art approaches in lip-reading sentences,the proposed system has demonstrated an improved performance by a 10%lower word error rate on average under varying illumination ratios.