检索结果-内蒙古大学图书馆

Modernizing Historical Documents: a User Study

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Domingo, Miguel Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain

Accessibility to historical documents is mostly limited to scholars. This is due to the language barrier inherent in human language and the linguistic properties of these documents. Given a historical document, modernization aims to generate a new version of it, written in the modern version of the document’s language. Its goal is to tackle the language barrier, decreasing the comprehension difficulty and making historical documents accessible to a broader audience. In this work, we proposed a new neural machine translation approach that profits from modern documents to enrich its systems. We tested this approach with both automatic and human evaluation, and conducted a user study. Results showed that modernization is successfully reaching its goal, although it still has room for improvement. Copyright © 2019, The Authors. All rights reserved.

关键词： History

Online learning for effort reduction in interactive neural machine translation

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Peris, Álvaro Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain

Neural machine translation systems require large amounts of training data and resources. Even with this, the quality of the translations may be insufficient for some users or domains. In such cases, the output of the system must be revised by a human agent. This can be done in a post-editing stage or following an interactive machine translation protocol. We explore the incremental update of neural machine translation systems during the post-editing or interactive translation processes. Such modifications aim to incorporate the new knowledge, from the edited sentences, into the translation system. Updates to the model are performed on-the-fly, as sentences are corrected, via online learning techniques. In addition, we implement a novel interactive, adaptive system, able to react to single-character interactions. This system greatly reduces the human effort required for obtaining high-quality translations. In order to stress our proposals, we conduct exhaustive experiments varying the amount and type of data available for training. Results show that online learning effectively achieves the objective of reducing the human effort required during the post-editing or the interactive machine translation stages. Moreover, these adaptive systems also perform well in scenarios with scarce resources. We show that a neural machine translation system can be rapidly adapted to a specific domain, exclusively by means of online learning techniques. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep learning

Tailored Design of Audio-Visual Speech recognition Models using Branchformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gimeno-Gómez, David Martínez-Hinarejos, Carlos D. Pattern Recognition and Human Language Technology research center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

Recent advances in Audio-Visual Speech recognition (AVSR) have led to unprecedented achievements in the field, improving the robustness of this type of system in adverse, noisy environments. In most cases, this task has been addressed through the design of models composed of two independent encoders, each dedicated to a specific modality. However, while recent works have explored unified audio-visual encoders, determining the optimal cross-modal architecture remains an ongoing challenge. Furthermore, such approaches often rely on models comprising vast amounts of parameters and high computational cost training processes. In this paper, we aim to bridge this research gap by introducing a novel audio-visual framework. Our proposed method constitutes, to the best of our knowledge, the first attempt to harness the flexibility and interpretability offered by encoder architectures, such as the Branchformer, in the design of parameter-efficient AVSR systems. To be more precise, the proposed framework consists of two steps: first, estimating audio- and video-only systems, and then designing a tailored audiovisual unified encoder based on the layer-level branch scores provided by the modality-specific models. Extensive experiments on English and Spanish AVSR benchmarks covering multiple data conditions and scenarios demonstrated the effectiveness of our proposed method. Even when trained on a moderate scale of data, our models achieve competitive word error rates (WER) of approximately 2.5% for English and surpass existing approaches for Spanish, establishing a new benchmark with an average WER of around 9.1%. These results reflect how our tailored AVSR system is able to reach state-of-the-art recognition rates while significantly reducing the model complexity w.r.t. the prevalent approach in the field. Code and pre-trained models are available at https://***/david-gimeno/tailored-avsr. © 2024, CC BY-NC-ND.

关键词： Speech enhancement

AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies 30

学校读者我要写书评

暂无评论

AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Vis...

Joint 30th International Conference on Computational Linguistics and 14th International Conference on language Resources and Evaluation, LREC-COLING 2024

作者： Acosta-Triana, José M. Gimeno-Gómez, David Martínez-Hinarejos, Carlos D. ValgrAI - Valencian Graduate School and Research Network of Artificial Intelligence Camino de Vera s/n 3Q Building València46022 Spain Pattern Recognition and Human Language Technologies Research Center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

ISBN: (纸本)9782493814104

More than 7,000 known languages are spoken around the world. However, due to the lack of annotated resources, only a small fraction of them are currently covered by speech technologies. Albeit self-supervised speech representations, recent massive speech corpora collections, as well as the organization of challenges, have alleviated this inequality, most studies are mainly benchmarked on English. This situation is aggravated when tasks involving both acoustic and visual speech modalities are addressed. In order to promote research on low-resource languages for audio-visual speech technologies, we present AnnoTheia, a semi-automatic annotation toolkit that detects when a person speaks on the scene and the corresponding transcription. In addition, to show the complete process of preparing AnnoTheia for a language of interest, we also describe the adaptation of a pre-trained model for active speaker detection to Spanish, using a database not initially conceived for this type of task. The AnnoTheia toolkit, tutorials, and pre-trained models are available at https://***/joactr/AnnoTheia/. © 2024 ELRA language Resource Association: CC BY-NC 4.0.

关键词： Speech recognition

Revisiting Bag-of-Word Metrics to Assess End-To-End Text Image recognition Results

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Toselli, Alejandro Héctor Vidal, Enrique Pattern Recognition and Human Language Technologies Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain

The accuracy of traditional, line-based text recognition systems depends only on how well the text in each given line image is recognized. But in recent approaches for end-to-end full-page image transcription, performance is additionally affected by line detection errors and by the order in which lines (or words in general) are provided by the (maybe implicit) layout analysis. Methods based on "bag-of-words" (BoW) have been used in the past to overcome the impact of word-order mismatch, which often leads the traditional edit-distance-based word error rate (WER) to yield pessimistic results. But BoW-based evaluation is not as popular as WER because it is presumed to provide optimistic word error estimates. We revisit the BoW idea and formally introduce evaluation metrics which are tightly and faithfully comparable with WER. We prove properties that qualify these metrics as reliably measures of order-independent word recognition accuracy, and analyse potential causes which may lead them to yield optimistic values. Finally, we report empirical results that favorably compare these metrics with the classical WER for several text recognition approaches and datasets. © 2023, The Authors. All rights reserved.

关键词： Optical character recognition

Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Maroñas, Juan Paredes, Roberto Ramos, Daniel PRHLT - Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de Valencia Spain AUDIAS - Audio Data Intelligence and Speech Universidad Autónoma de Madrid Spain

Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage, implemented with a Bayesian Neural Network (BNN), to map the uncalibrated probabilities provided by a DNN to calibrated ones, consistently improving calibration. Our results evidence that incorporating uncertainty provides more reliable probabilistic models, a critical condition for achieving good calibration. We report a generous collection of experimental results using high-accuracy DNNs in standardized image classification benchmarks, showing the good performance, flexibility and robust behavior of our approach with respect to several state-of-the-art calibration methods. Code for reproducibility is provided. Copyright © 2019, The Authors. All rights reserved.

关键词： Calibration

Speaker-Adapted End-to-End Visual Speech recognition for Continuous Spanish

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gimeno-Gómez, David Martínez-Hinarejos, Carlos-D Pattern Recognition and Human Language Technologies Research Center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

Different studies have shown the importance of visual cues throughout the speech perception process. In fact, the development of audiovisual approaches has led to advances in the field of speech technologies. However, although noticeable results have recently been achieved, visual speech recognition remains an open research problem. It is a task in which, by dispensing with the auditory sense, challenges such as visual ambiguities and the complexity of modeling silence must be faced. Nonetheless, some of these challenges can be alleviated when the problem is approached from a speaker-dependent perspective. Thus, this paper studies, using the Spanish LIP-RTVE database, how the estimation of specialized end-to-end systems for a specific person could affect the quality of speech recognition. First, different adaptation strategies based on the fine-tuning technique were proposed. Then, a pre-trained CTC/Attention architecture was used as a baseline throughout our experiments. Our findings showed that a two-step fine-tuning process, where the VSR system is first adapted to the task domain, provided significant improvements when the speaker adaptation was addressed. Furthermore, results comparable to the current state of the art were reached even when only a limited amount of data was available. © 2023, CC BY-NC-SA.

关键词： Speech recognition

Analysis of Visual Features for Continuous Lipreading in Spanish

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gimeno-Gómez, David Martínez-Hinarejos, Carlos D. Pattern Recognition and Human Language Technologies Research Center Universitat Politècnica de València Camino de Vera s/n València46022 Spain

During a conversation, our brain is responsible for combining information obtained from multiple senses in order to improve our ability to understand the message we are perceiving. Different studies have shown the importance of presenting visual information in these situations. Nevertheless, lipreading is a complex task whose objective is to interpret speech when audio is not available. By dispensing with a sense as crucial as hearing, it will be necessary to be aware of the challenge that this lack presents. In this paper, we propose an analysis of different speech visual features with the intention of identifying which of them is the best approach to capture the nature of lip movements for natural Spanish and, in this way, dealing with the automatic visual speech recognition task. In order to estimate our system, we present an audiovisual corpus compiled from a subset of the RTVE database, which has been used in the Albayzín evaluations. We employ a traditional system based on Hidden Markov Models with Gaussian Mixture Models. Results show that, although the task is difficult, in restricted conditions we obtain recognition results which determine that using eigenlips in combination with deep features is the best visual approach. © 2023, CC BY-NC-SA.

关键词： Hidden Markov models

LIP-RTVE: An Audiovisual Database for Continuous Spanish in the Wild

学校读者我要写书评

暂无评论

arXiv 2023年

Speech is considered as a multi-modal process where hearing and vision are two fundamentals pillars. In fact, several studies have demonstrated that the robustness of Automatic Speech recognition systems can be improved when audio and visual cues are combined to represent the nature of speech. In addition, Visual Speech recognition, an open research problem whose purpose is to interpret speech by reading the lips of the speaker, has been a focus of interest in the last decades. Nevertheless, in order to estimate these systems in the currently Deep Learning era, large-scale databases are required. On the other hand, while most of these databases are dedicated to English, other languages lack sufficient resources. Thus, this paper presents a semi-automatically annotated audiovisual database to deal with unconstrained natural Spanish, providing 13 hours of data extracted from Spanish television. Furthermore, baseline results for both speaker-dependent and speaker-independent scenarios are reported using Hidden Markov Models, a traditional paradigm that has been widely used in the field of Speech Technologies. © 2023, CC BY-NC-SA.

关键词： Speech recognition