检索结果-内蒙古大学图书馆

Tailored Design of Audio-Visual Speech recognition Models using Branchformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gimeno-Gómez, David Martínez-Hinarejos, Carlos D. Pattern Recognition and Human Language Technology research center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

Recent advances in Audio-Visual Speech recognition (AVSR) have led to unprecedented achievements in the field, improving the robustness of this type of system in adverse, noisy environments. In most cases, this task has been addressed through the design of models composed of two independent encoders, each dedicated to a specific modality. However, while recent works have explored unified audio-visual encoders, determining the optimal cross-modal architecture remains an ongoing challenge. Furthermore, such approaches often rely on models comprising vast amounts of parameters and high computational cost training processes. In this paper, we aim to bridge this research gap by introducing a novel audio-visual framework. Our proposed method constitutes, to the best of our knowledge, the first attempt to harness the flexibility and interpretability offered by encoder architectures, such as the Branchformer, in the design of parameter-efficient AVSR systems. To be more precise, the proposed framework consists of two steps: first, estimating audio- and video-only systems, and then designing a tailored audiovisual unified encoder based on the layer-level branch scores provided by the modality-specific models. Extensive experiments on English and Spanish AVSR benchmarks covering multiple data conditions and scenarios demonstrated the effectiveness of our proposed method. Even when trained on a moderate scale of data, our models achieve competitive word error rates (WER) of approximately 2.5% for English and surpass existing approaches for Spanish, establishing a new benchmark with an average WER of around 9.1%. These results reflect how our tailored AVSR system is able to reach state-of-the-art recognition rates while significantly reducing the model complexity w.r.t. the prevalent approach in the field. Code and pre-trained models are available at https://***/david-gimeno/tailored-avsr. © 2024, CC BY-NC-ND.

关键词： Speech enhancement

A comparative large scale study of MLP features for mandarin ASR

学校读者我要写书评

暂无评论

A comparative large scale study of MLP features for mandarin...

作者： Valente, Fabio Doss, Mathew Magimai Plahl, Christian Ravuri, Suman Wang, Wen IDIAP Research Institute CH-1920 Martigny Switzerland Human Language Technology and Pattern Recognition RWTH Aachen University Germany International Computer Science Institute 1947 Center Street Berkeley CA 94704 United States Speech Technology and Research Laboratory SRI International Menlo Park CA United States

MLP based front-ends have shown significant complementary properties to conventional spectral features. As part of the DARPA GALE program, different MLP features were developed for Mandarin ASR. In this paper, all the proposed frontends are compared in systematic manner and we extensively investigate the scalability of these features in terms of the amount of training data (from 100 hours to 1600 hours) and system complexity (maximum likelihood training, SAT, lattice level combination, and discriminative training). Results on 5 hours of evaluation data from the GALE project reveal that the MLP features consistently produce relative improvements in the range of 15% - 23% at the different steps of a multipass system when compared to the conventional short-term spectral based features like MFCC and PLP. The largest improvement is obtained using a hierarchical MLP approach. © 2010 ISCA.

关键词： Maximum likelihood

How much does Tokenization affect neural machine translation?

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Domingo, Miguel García-Martínez, Mercedes Helle, Alexandre Casacuberta, Francisco Herranz, Manuel Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain Pangeanic / B.I Europa PangeaMT Technologies Division Valencia Spain

Tokenization or segmentation is a wide concept that covers simple processes such as separating punctuation from words, or more sophisticated processes such as applying morphological knowledge. Neural Machine Translation (NMT) requires a limited-size vocabulary for computational cost and enough examples to estimate word embeddings. Separating punctuation and splitting tokens into words or subwords has proven to be helpful to reduce vocabulary and increase the number of examples of each word, improving the translation quality. Tokenization is more challenging when dealing with languages with no separator between words. In order to assess the impact of the tokenization in the quality of the final translation on NMT, we experimented on five tokenizers over ten language pairs. We reached the conclusion that the tokenization significantly affects the final translation quality and that the best tokenizer differs for different language pairs. Copyright © 2018, The Authors. All rights reserved.

关键词： Neural machine translation

Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Maroñas, Juan Paredes, Roberto Ramos, Daniel PRHLT - Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de Valencia Spain AUDIAS - Audio Data Intelligence and Speech Universidad Autónoma de Madrid Spain

Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage, implemented with a Bayesian Neural Network (BNN), to map the uncalibrated probabilities provided by a DNN to calibrated ones, consistently improving calibration. Our results evidence that incorporating uncertainty provides more reliable probabilistic models, a critical condition for achieving good calibration. We report a generous collection of experimental results using high-accuracy DNNs in standardized image classification benchmarks, showing the good performance, flexibility and robust behavior of our approach with respect to several state-of-the-art calibration methods. Code for reproducibility is provided. Copyright © 2019, The Authors. All rights reserved.

关键词： Calibration

Sample drop detection for distant-speech recognition with asynchronous devices distributed in space

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Raissi, Tina Pascual, Santiago Omologo, Maurizio Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany Universitat Politècnica de Catalunya Barcelona Spain Center for Information and Communication Technology Fondazione Bruno Kessler Trento Italy

In many applications of multi-microphone multi-device processing, the synchronization among different input channels can be affected by the lack of a common clock and isolated drops of samples. In this work, we address the issue of sample drop detection in the context of a conversational speech scenario, recorded by a set of microphones distributed in space. The goal is to design a neural-based model that given a short window in the time domain, detects whether one or more devices have been subjected to a sample drop event. The candidate time windows are selected from a set of large time intervals, possibly including a sample drop, and by using a preprocessing step. The latter is based on the application of normalized cross-correlation between signals acquired by different devices. The architecture of the neural network relies on a CNN-LSTM encoder, followed by multi-head attention. The experiments are conducted using both artificial and real data. Our proposed approach obtained F1 score of 88% on an evaluation set extracted from the CHiME-5 corpus. A comparable performance was found in a larger set of experiments conducted on a set of multi-channel artificial scenes. Copyright © 2019, The Authors. All rights reserved.

关键词： Speech recognition

ICFHR2016 Handwritten Keyword Spotting Competition (H-KWS 2016)

学校读者我要写书评

暂无评论

ICFHR2016 Handwritten Keyword Spotting Competition (H-KWS 20...

International Workshop on Frontiers in Handwriting recognition

作者： Ioannis Pratikakis Konstantinos Zagoris Basilis Gatos Joan Puigcerver Alejandro H. Toselli Enrique Vidal Visual Computing Group Democritus University of Thrace Xanthi Greece Institute of Informatics and Telecommunications National Center for Scientific Research “Demokritos” Athens Greece Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Valencia Spain

ISBN: (纸本)9781509009824

The H-KWS 2016, organized in the context of the ICFHR 2016 conference aims at setting up an evaluation framework for benchmarking handwritten keyword spotting (KWS) examining both the Query by Example (QbE) and the Query by String (QbS) approaches. Both KWS approaches were hosted into two different tracks, which in turn were split into two distinct challenges, namely, a segmentation-based and a segmentation-free to accommodate different perspectives adopted by researchers in the KWS field. In addition, the competition aims to evaluate the submitted training-based methods under different amounts of training data. Four participants submitted at least one solution to one of the challenges, according to the capabilities and/or restrictions of their systems. The data used in the competition consisted of historical German and English documents with their own characteristics and complexities. This paper presents the details of the competition, including the data, evaluation metrics and results of the best run of each participating methods.

关键词： Image segmentation Training data Training Measurement Histograms Handwriting recognition Visualization

Automatic Semantic Segmentation of Structural Elements related to the Spinal Cord in the Lumbar Region by using Convolutional Neural Networks

学校读者我要写书评

暂无评论

Automatic Semantic Segmentation of Structural Elements relat...

International Conference on pattern recognition

作者： Jhon Jairo Saenz-Gamboa Maria de la Iglesia-Vayá Jon A. Gómez Biomedical Imaging Joint Unit Foundation for the Promotion of Health and Biomedical Research FISABIO-CIPF València Spain Pattern Recognition and Human Language Technology research center Universitat Politècnica de València València Spain

This work addresses the problem of automatically segmenting the MR images corresponding to the lumbar spine. The purpose is to detect and delimit the different structural elements like vertebrae, intervertebral discs, nerves, blood vessels, etc. This task is known as semantic segmentation. The approach proposed in this work is based on convolutional neural networks whose output is a mask where each pixel from the input image is classified into one of the possible classes. Classes were defined by radiologists and correspond to structural elements and tissues. The proposed network architectures are variants of the U-Net. Several complementary blocks were used to define the variants: spatial attention models, deep supervision and multi-kernels at input, this last block type is based on the idea of inception. Those architectures which got the best results are described in this paper, and their results are discussed. Two of the proposed architectures outperform the standard U-Net used as baseline. Those architectures which got the best results are described in this paper, and their results are discussed. Two of the proposed architectures outperform the standard U-Net used as baseline.

关键词： Image segmentation Semantics Spine Morphology Computer architecture Blood vessels Fats

Bridging the Native language and language Variety Identification Tasks

学校读者我要写书评

暂无评论

Procedia Computer Science 2017年 112卷 1554-1561页

作者： Marc Franco-Salvador Greg Kondrak Paolo Rosso Symanto Research 90425 Nuremberg Germany Pattern Recognition and Human Language Technology (PRHLT) Research Center Universitat Politècnica de València 46022 Valencia Spain Department of Computing Science University of Alberta Edmonton AB T6G 2E8 Canada

The objective of Native language Identification is to determine the native language of the author of a text that he or she wrote in another language. By contrast, language Variety Identification aims at classifying texts representing different varieties of a single language. We postulate that both tasks may be reduced to a single objective, which is to identify the language variety of the text. We design a general approach that combines string kernels and word embeddings, which capture different characteristics of texts. The results of our experiments show that the approach achieves excellent results on both tasks, without any task-specific adaptations.

关键词： Native language Identification language Variety Identification String Kernels Word Embeddings Classifier Combination

AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech technologies

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Acosta-Triana, José M. Gimeno-Gómez, David Martínez-Hinarejos, Carlos D. ValgrAI - Valencian Graduate School Research Network of Artificial Intelligence Camino de Vera s/n 3Q Building València46022 Spain Pattern Recognition and Human Language Technologies Research Center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

More than 7,000 known languages are spoken around the world. However, due to the lack of annotated resources, only a small fraction of them are currently covered by speech technologies. Albeit self-supervised speech representations, recent massive speech corpora collections, as well as the organization of challenges, have alleviated this inequality, most studies are mainly benchmarked on English. This situation is aggravated when tasks involving both acoustic and visual speech modalities are addressed. In order to promote research on low-resource languages for audio-visual speech technologies, we present AnnoTheia, a semi-automatic annotation toolkit that detects when a person speaks on the scene and the corresponding transcription. In addition, to show the complete process of preparing AnnoTheia for a language of interest, we also describe the adaptation of a pre-trained model for active speaker detection to Spanish, using a database not initially conceived for this type of task. The AnnoTheia toolkit, tutorials, and pre-trained models are available at https://***/joactr/AnnoTheia/. © 2024, CC BY-NC-ND.

关键词： Speech recognition