检索结果-内蒙古大学图书馆

5th Conference on Machine Translation, WMT 2020

作者： Huo, Jingjing Herold, Christian Gao, Yingbo Dahlmann, Leonard Khadivi, Shahram Ney, Hermann eBay Inc. Aachen Germany Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (纸本)9781948087810

Context-aware neural machine translation (NMT) is a promising direction to improve the translation quality by making use of the additional context, e.g., document-level translation, or having meta-information. Although there exist various architectures and analyses, the effectiveness of different context-aware NMT models is not well explored yet. This paper analyzes the performance of document-level NMT models on four diverse domains with a varied amount of parallel document-level bilingual data. We conduct a comprehensive set of experiments to investigate the impact of document-level NMT. We find that there is no single best approach to document-level NMT, but rather that different architectures come out on top on different tasks. Looking at task-specific problems, such as pronoun resolution or headline translation, we find improvements in the context-aware systems, even in cases where the corpus-level metrics like BLEU show no significant improvement. We also show that document-level back-translation significantly helps to compensate for the lack of document-level bi-texts. © 2020 Association for Computational Linguistics

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Classification Error Bound for Low Bayes Error Conditions in Machine Learning

Classification Error Bound for Low Bayes Error Conditions in...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Zijian Yang Vahe Eminyan Ralf Schlüter Hermann Ney Computer Science Department Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 RWTH Aachen University Germany AppTek GmbH Germany

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

In statistical classification and machine learning, classification error is an important performance measure, which is minimized by the Bayes decision rule. In practice, the unknown true distribution is usually replaced with a model distribution estimated from the training data in the Bayes decision rule. This substitution introduces a mismatch between the Bayes error and the model-based classification error. In this work, we apply classification error bounds to study the relationship between the error mismatch and the Kullback-Leibler divergence in machine learning. Motivated by recent observations of low model-based classification errors in many machine learning tasks, bounding the Bayes error to be lower, we propose a linear approximation of the classification error bound for low Bayes error conditions. Then, the bound for class priors are discussed. Moreover, we extend the classification error bound for sequences. Using automatic speech recognition as a representative example of machine learning applications, this work analytically discusses the correlations among different performance measures with extended bounds, including cross-entropy loss, language model perplexity, and word error rate.

关键词： Error analysis Measurement uncertainty Linear approximation Training data Machine learning Signal processing Loss measurement Data models Bayes methods Speech processing

来源：评论

学校读者我要写书评

暂无评论

Classification Error Bound for Low Bayes Error Conditions in Machine Learning

arXiv

引用

arXiv 2025年

作者： Yang, Zijian Eminyan, Vahe Schlüter, Ralf Ney, Hermann Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 Computer Science Department RWTH Aachen University Germany AppTek GmbH Germany

关键词： Speech recognition

来源：评论

学校读者我要写书评

暂无评论

Unifying Input and Output Smoothing in Neural Machine Translation 28

Unifying Input and Output Smoothing in Neural Machine Transl...

引用

28th International Conference on Computational Linguistics, COLING 2020

作者： Gao, Yingbo Liao, Baohao Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

ISBN: (纸本)9781952148279

Soft contextualized data augmentation is a recent method that replaces one-hot representation of words with soft posterior distributions of an external language model, smoothing the input of neural machine translation systems. Label smoothing is another effective method that penalizes over-confident model outputs by discounting some probability mass from the true target word, smoothing the output of neural machine translation systems. Having the benefit of updating all word vectors in each optimization step and better regularizing the models, the two smoothing methods are shown to bring significant improvements in translation performance. In this work, we study how to best combine the methods and stack the improvements. Specifically, we vary the prior distributions to smooth with, the hyperparameters that control the smoothing strength, and the token selection procedures. We conduct extensive experiments on small datasets, evaluate the recipes on larger datasets, and examine the implications when back-translation is further used. Our results confirm cumulative improvements when input and output smoothing are used in combination, giving up to +1.9 BLEU scores on standard machine translation tasks and reveal reasons why these smoothing methods should be preferred. © 2020 COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference. All rights reserved.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Two-Way Neural Machine Translation: A Proof of Concept for Bidirectional Translation Modeling Using a Two-Dimensional Grid

Two-Way Neural Machine Translation: A Proof of Concept for B...

引用

IEEE Spoken language technology Workshop

作者： Parnia Bahar Christopher Brix Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany AppTek GmbH Aachen Germany

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

Neural translation models have proven to be effective in capturing sufficient information from a source sentence and generating a high-quality target sentence. However, it is not easy to get the best effect for bidirectional translation, i.e., both source-to-target and target-to-source translation using a single model. If we exclude some pioneering attempts, such as multilingual systems, all other bidirectional translation approaches are required to train two individual models. This paper proposes to build a single end-to-end bidirectional translation model using a two-dimensional grid, where the left-to-right decoding generates source-to-target, and the bottom-to-up decoding creates target-to-source output. Instead of training two models independently, our approach encourages a single network to jointly learn to translate in both directions. Experiments on the WMT2018 German↔English and Turkish↔English translation tasks show that the proposed model is capable of generating a good translation quality and has sufficient potential to direct the research.

关键词： Training Conferences Decoding Machine translation Task analysis

来源：评论

学校读者我要写书评

暂无评论

Tight Integrated End-to-End Training for Cascaded Speech Translation

Tight Integrated End-to-End Training for Cascaded Speech Tra...

引用

IEEE Spoken language technology Workshop

作者： Parnia Bahar Tobias Bieschke Ralf Schlüter Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany AppTek GmbH Aachen Germany

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

A cascaded speech translation model relies on discrete and non-differentiable transcription, which provides a supervision signal from the source side and helps the transformation between source speech and target text. Such modeling suffers from error propagation between ASR and MT models. Direct speech translation is an alternative method to avoid error propagation; however, its performance is often behind the cascade system. To use an intermediate representation and preserve the end-to-end trainability, previous studies have proposed using two-stage models by passing the hidden vectors of the recognizer into the decoder of the MT model and ignoring the MT encoder. This work explores the feasibility of collapsing the entire cascade components into a single end-to-end trainable model by optimizing all parameters of ASR and MT models jointly without ignoring any learned parameters. It is a tightly integrated method that passes renormalized source word posterior distributions as a soft decision instead of one-hot vectors and enables backpropagation. Therefore, it provides both transcriptions and translations and achieves strong consistency between them. Our experiments on four tasks with different data scenarios show that the model outperforms cascade models up to 1.8% in BLEU and 2.0% in TER and is superior compared to direct models.

关键词： Training Adaptation models Correlation Training data Data models Decoding Task analysis

来源：评论

学校读者我要写书评

暂无评论

On Architectures and Training for Raw Waveform Feature Extraction in ASR

On Architectures and Training for Raw Waveform Feature Extra...

引用

IEEE Workshop on Automatic Speech recognition and Understanding

作者： Peter Vieting Christoph Lüscher Wilfried Michel Ralf Schlüter Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany AppTek GmbH Aachen Germany

ISBN: (纸本)9781665437400

With the success of neural network based modeling in auto-matic speech recognition (ASR), many studies investigated acoustic modeling and learning of feature extractors directly based on the raw waveform. Recently, one line of research has focused on unsupervised pre-training of feature extractors on audio-only data to improve downstream ASR performance. In this work, we investigate the usefulness of one of these front-end frameworks, namely wav2vec, in a setting without additional untranscribed data for hybrid ASR systems. We compare this framework both to the manually defined stan-dard Gammatone feature set, as well as to features extracted as part of the acoustic model of an ASR system trained su-pervised. We study the benefits of using the pre-trained feature extractor and explore how to additionally exploit an ex-isting acoustic model trained with different features. Finally, we systematically examine combinations of the described features in order to further advance the performance.

关键词： Training Conferences Neural networks Feature extraction Acoustics Data mining Automatic speech recognition

来源：评论

学校读者我要写书评

暂无评论

Adapting document-grounded dialog systems to spoken conversations using data augmentation and a noisy channel model

arXiv

引用

arXiv 2021年

作者： Thulke, David Daheim, Nico Dugast, Christian Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Germany AppTek GmbH Aachen Germany

This paper summarizes our submission to Task 2 of the second track of the 10th Dialog System technology Challenge (DSTC10) "Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations". Similar to the previous year's iteration, the task consists of three subtasks: detecting whether a turn is knowledge seeking, selecting the relevant knowledge document and finally generating a grounded response. This year, the focus lies on adapting the system to noisy ASR transcripts. We explore different approaches to make the models more robust to this type of input and to adapt the generated responses to the style of spoken conversations. For the latter, we get the best results with a noisy channel model that additionally reduces the number of short and generic responses. Our best system achieved the 1st rank in the automatic and the 3rd rank in the human evaluation of the challenge. Copyright © 2021, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Refined Statistical Bounds for Classification Error Mismatches with Constrained Bayes Error

Refined Statistical Bounds for Classification Error Mismatch...

引用

IEEE (first ITW conference of the year) Information Theory Workshop (ITW)

ISBN: (数字)9798350348934

ISBN: (纸本)9798350348941

In statistical classification/multiple hypothesis testing and machine learning, a model distribution estimated from the training data is usually applied to replace the unknown true distribution in the Bayes decision rule, which introduces a mismatch between the Bayes error and the model-based classification error. In this work, we derive the classification error bound to study the relationship between the Kullback-Leibler divergence and the classification error mismatch. We first reconsider the statistical bounds based on classification error mismatch derived in previous works, employing a different method of derivation. Then, motivated by the observation that the Bayes error is typically low in machine learning tasks like speech recognition and pattern recognition, we derive a refined Kullback-Leibler-divergence-based bound on the error mismatch with the constraint that the Bayes error is lower than a threshold.

关键词： Analytical models Conferences Training data Machine learning Speech recognition Data models Bayes methods Testing Information theory

来源：评论

学校读者我要写书评

暂无评论

Towards consistent hybrid HMM acoustic modeling

arXiv

引用

arXiv 2021年

作者： Raissi, Tina Beck, Eugen Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Germany AppTek GmbH Aachen Germany

High-performance hybrid automatic speech recognition (ASR) systems are often trained with clustered triphone outputs, and thus require a complex training pipeline to generate the clustering. The same complex pipeline is often utilized in order to generate an alignment for use in frame-wise cross-entropy training. In this work, we propose a flat-start factored hybrid model trained by modeling the full set of triphone states explicitly without relying on clustering methods. This greatly simplifies the training of new models. Furthermore, we study the effect of different alignments used for Viterbi training. Our proposed models achieve competitive performance on the Switchboard task compared to systems using clustered triphones and other flat-start models in the *** Codes 68T10 Copyright © 2021, The Authors. All rights reserved.

关键词： Pipelines

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：