检索结果-内蒙古大学图书馆

Enhancing and Adversarial: Improve ASR with Speaker Labels

学校读者我要写书评

暂无评论

Enhancing and Adversarial: Improve ASR with Speaker Labels

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wei Zhou Haotian Wu Jingjing Xu Mohammad Zeineldeen Christoph Lüscher Ralf Schlüter Hermann Ney Computer Science Department Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany AppTek GmbH Aachen Germany

ASR can be improved by multi-task learning (MTL) with domain enhancing or domain adversarial training, which are two opposite objectives with the aim to increase/decrease domain variance towards domain-aware/agnostic ASR, respectively. In this work, we study how to best apply these two opposite objectives with speaker labels to improve conformer-based ASR. We also propose a novel adaptive gradient reversal layer for stable and effective adversarial training without tuning effort. Detailed analysis and experimental verification are conducted to show the optimal positions in the ASR neural network (NN) to apply speaker enhancing and adversarial training. We also explore their combination for further improvement, achieving the same performance as i-vectors plus adversarial training. Our best speaker-based MTL achieves 7% relative improvement on the Switchboard Hub5’00 set. We also investigate the effect of such speaker-based MTL w.r.t. cleaner dataset and weaker ASR NN.

关键词： Training Artificial neural networks Switches Signal processing Multitasking Acoustics Speech processing

Towards a Better Evaluation of Metrics for Machine Translation 5

学校读者我要写书评

暂无评论

Towards a Better Evaluation of Metrics for Machine Translati...

5th Conference on Machine Translation, WMT 2020

作者： Stanchev, Peter Wang, Weiyue Ney, Hermann Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University Aachen52056 Germany

ISBN: (纸本)9781948087810

An important aspect of machine translation is its evaluation, which can be achieved through the use of a variety of metrics. To compare these metrics, the workshop on statistical machine translation annually evaluates metrics based on their correlation with human judgement. Over the years, methods for measuring correlation with humans have changed, but little research has been performed on what the optimal methods for acquiring human scores are and how human correlation can be measured. In this work, the methods for evaluating metrics at both system- and segment-level are analyzed in detail and their shortcomings are pointed out. © 2020 Association for Computational Linguistics

关键词： Machine translation

Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization

学校读者我要写书评

暂无评论

Prompting and Fine-Tuning of Small LLMs for Length-Controlla...

Foundation and Large language Models (FLLM), International Conference on

作者： David Thulke Yingbo Gao Rricha Jalota Christian Dugast Hermann Ney AppTek GmbH Aachen Machine Learning and Human Language Technology Group RWTH Aachen University

ISBN: (数字)9798350354799

ISBN: (纸本)9798350354805

This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting existing LLMs to generate summaries of telephone conversations, followed by the creation of a tailored synthetic training dataset utilizing stronger frontier models. We place special focus on the diversity of the generated data and on the ability to control the length of the generated summaries to meet various use-case specific requirements. The effectiveness of our method is evaluated using two state-of-the-art LLM-as-a-judge-based evaluation techniques to ensure the quality and relevance of the summaries. Our results show that fine-tuned Llama-2-7B-based summarization model performs on-par with GPT-4 in terms of factual accuracy, completeness and conciseness. Our findings demonstrate the potential for quickly bootstrapping a practical and efficient call summarization system.

关键词： Training Measurement Accuracy Large language models Speech recognition Oral communication Telephone sets Data models Reliability Synthetic data

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

学校读者我要写书评

暂无评论

Controllable Factuality in Document-Grounded Dialog Systems ...

2022 Findings of the Association for Computational Linguistics: EMNLP 2022

作者： Daheim, Nico Thulke, David Dugast, Christian Ney, Hermann Ubiquitous Knowledge Processing Lab Department of Computer Science Technical University of Darmstadt Germany Human Language Technology and Pattern Recognition RWTH Aachen University Germany AppTek GmbH

In this work, we present a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes' theorem. One component is a traditional ungrounded response generation model and the other component models the reconstruction of the grounding document based on the dialog context and generated response. We propose different approximate decoding schemes and evaluate our approach on multiple open-domain and task-oriented document-grounded dialog datasets. Our experiments show that the model is more factual in terms of automatic factuality metrics than the baseline model. Furthermore, we outline how introducing scaling factors between the components allows for controlling the tradeoff between factuality and fluency in the model output. Finally, we compare our approach to a recently proposed method to control factuality in grounded dialog, CTRL (Rashkin et al., 2021), and show that both approaches can be combined to achieve additional improvements. © 2022 Association for Computational Linguistics.

关键词： Computational linguistics

Robust Knowledge Distillation from RNN-T Models with Noisy Training Labels Using Full-Sum Loss

学校读者我要写书评

暂无评论

Robust Knowledge Distillation from RNN-T Models with Noisy T...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Mohammad Zeineldeen Kartik Audhkhasi Murali Karthick Baskar Bhuvana Ramabhadran Computer Science Department Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany Google LLC New York

This work studies knowledge distillation (KD) and addresses its constraints for recurrent neural network transducer (RNN-T) models. In hard distillation, a teacher model transcribes large amounts of unlabelled speech to train a student model. Soft distillation is another popular KD method that distills the output logits of the teacher model. Due to the nature of RNN-T alignments, applying soft distillation between RNNT architectures having different posterior distributions is challenging. In addition, bad teachers having high word-error-rate (WER) reduce the efficacy of KD. We investigate how to effectively distill knowledge from variable quality ASR teachers, which has not been studied before to the best of our knowledge. We show that a sequence-level KD, full-sum distillation, outperforms other distillation methods for RNN-T models, especially for bad teachers. We also propose a variant of full-sum distillation that distills the sequence discriminative knowledge of the teacher leading to further improvement in WER. We conduct experiments on public datasets namely SpeechStew and LibriSpeech, and on in-house production data.

关键词： Knowledge engineering Training Recurrent neural networks Transducers Production Signal processing Acoustics

ON THE RELATION BETWEEN INTERNAL language MODEL AND SEQUENCE DISCRIMINATIVE TRAINING FOR NEURAL TRANSDUCERS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yang, Zijian Zhou, Wei Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University Aachen52074 Germany AppTek GmbH Aachen52062 Germany

Internal language model (ILM) subtraction has been widely applied to improve the performance of the RNN-Transducer with external language model (LM) fusion for speech recognition. In this work, we show that sequence discriminative training has a strong correlation with ILM subtraction from both theoretical and empirical points of view. Theoretically, we derive that the global optimum of maximum mutual information (MMI) training shares a similar formula as ILM subtraction. Empirically, we show that ILM subtraction and sequence discriminative training achieve similar effects across a wide range of experiments on Librispeech, including both MMI and minimum Bayes risk (MBR) criteria, as well as neural transducers and LMs of both full and limited context. The benefit of ILM subtraction also becomes much smaller after sequence discriminative training. We also provide an in-depth study to show that sequence discriminative training has a minimal effect on the commonly used zero-encoder ILM estimation, but a joint effect on both encoder and prediction + joint network for posterior probability reshaping including both ILM and blank suppression. Copyright © 2023, The Authors. All rights reserved.

关键词： Transducers

HMM VS. CTC FOR AUTOMATIC SPEECH recognition: COMPARISON BASED ON FULL-SUM TRAINING FROM SCRATCH

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Raissi, Tina Zhou, Wei Berger, Simon Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Group Rwth Aachen University Germany AppTek GmbH Aachen Germany

In this work, we compare from-scratch sequence-level cross-entropy (full-sum) training of Hidden Markov Model (HMM) and Connectionist Temporal Classification (CTC) topologies for automatic speech recognition (ASR). Besides accuracy, we further analyze their capability for generating high-quality time alignment between the speech signal and the transcription, which can be crucial for many subsequent applications. Moreover, we propose several methods to improve convergence of from-scratch full-sum training by addressing the alignment modeling issue. Systematic comparison is conducted on both Switchboard and LibriSpeech corpora across CTC, posterior HMM with and w/o transition probabilities, and standard hybrid HMM. We also provide a detailed analysis of both Viterbi forced-alignment and Baum-Welch full-sum occupation probabilities. Copyright © 2022, The Authors. All rights reserved.

关键词： Hidden Markov models

IMPROVING FACTORED HYBRID HMM ACOUSTIC MODELING WITHOUT STATE TYING

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Raissi, Tina Beck, Eugen Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Germany AppTek GmbH Aachen Germany

In this work, we show that a factored hybrid hidden Markov model (FH-HMM) which is defined without any phonetic state-tying outperforms a state-of-the-art hybrid HMM. The factored hybrid HMM provides a link to transducer models in the way it models phonetic (label) context while preserving the strict separation of acoustic and language model of the hybrid HMM approach. Furthermore, we show that the factored hybrid model can be trained from scratch without using phonetic state-tying in any of the training steps. Our modeling approach enables triphone context while avoiding phonetic state-tying by a decomposition into locally normalized factored posteriors for monophones/HMM states in phoneme context. Experimental results are provided for Switchboard 300h and LibriSpeech. On the former task we also show that by avoiding the phonetic state-tying step, the factored hybrid can take better advantage of regularization techniques during training, compared to the standard hybrid HMM with phonetic state-tying based on classification and regression trees (CART).MSC Codes 68T10 Copyright © 2022, The Authors. All rights reserved.

关键词： Hidden Markov models

Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Thulke, David Gao, Yingbo Jalota, Rricha Dugast, Christian Ney, Hermann AppTek GmbH Aachen Germany Machine Learning and Human Language Technology Group RWTH Aachen University Germany

This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting existing LLMs to generate summaries of telephone conversations, followed by the creation of a tailored synthetic training dataset utilizing stronger frontier models. We place special focus on the diversity of the generated data and on the ability to control the length of the generated summaries to meet various use-case specific requirements. The effectiveness of our method is evaluated using two state-of-the-art LLM-as-a-judgebased evaluation techniques to ensure the quality and relevance of the summaries. Our results show that fine-tuned Llama-2- 7B-based summarization model performs on-par with GPT-4 in terms of factual accuracy, completeness and conciseness. Our findings demonstrate the potential for quickly bootstrapping a practical and efficient call summarization system. © 2024, CC BY-SA.

关键词： Modeling languages