检索结果-内蒙古大学图书馆

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Kazuki Irie Zhihong Lei Ralf Schlüter Hermann Ney Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University D-52056 Aachen Germany

ISBN: (纸本)9781538646595

Long short-term memory (LSTM) recurrent neural network language models compress the full context of variable lengths into a fixed size vector. In this work, we investigate the task of predicting the LSTM hidden representation of the full context from a truncated n-gram context as a subtask for training an n-gram feedforward language model. Since this approach is a form of knowledge distillation, we compare two methods. First, we investigate the standard transfer based on the Kullback-Leibler divergence of the output distribution of the feedforward model from that of the LSTM. Second, we minimize the mean squared error between the hidden state of the LSTM and that of the n-gram feedforward model. We carry out experiments on different subsets of the Switchboard speech recognition dataset for feedforward models with a short (5-gram) and a medium (10-gram) context length. We show that we get improvements in perplexity and word error rate of up to 8% and 4% relative for the medium model, while the improvements are only marginal for the short model.

关键词： language modeling neural networks knowledge distillation student-teacher speech recognition modelling languages Speech recognition FEEDFORWARD Neural network Error analysis

来源：评论

学校读者我要写书评

暂无评论

Unsupervised training for large vocabulary translation using sparse lexicon andword classes 15

Unsupervised training for large vocabulary translation using...

引用

15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017

作者： Kim, Yunsu Schamper, Julian Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Germany

ISBN: (纸本)9781510838604

We address for the first time unsupervised training for a translation task with hundreds of thousands of vocabulary words. We scale up the expectation-maximization (EM) algorithm to learn a large translation table without any parallel text or seed lexicon. First, we solve the memory bottleneck and enforce the sparsity with a simple thresholding scheme for the lexicon. Second, we initialize the lexicon training with word classes, which efficiently boosts the performance. Our methods produced promising results on two large-scale unsupervised translation tasks. © 2017 Association for Computational Linguistics.

关键词： Maximum principle

来源：评论

学校读者我要写书评

暂无评论

The rwth aachen university English-German and German-English machine translation system for WMT 2017 2

The RWTH Aachen University English-German and German-English...

引用

2nd Conference on Machine Translation, WMT 2017

作者： Peter, Jan-Thorsten Guta, Andreas Alkhouli, Tamer Bahar, Parnia Rosendahl, Jan Rossenbach, Nick Graça, Miguel Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

ISBN: (纸本)9781945626968

This paper describes the statistical machine translation system developed at rwth aachen university for the English?German and German?English translation tasks of the EMNLP 2017 Second Conference on Machine Translation (WMT 2017). We use ensembles of attention-based neural machine translation system for both directions. We use the provided parallel and synthetic data to train the models. In addition, we also create a phrasal system using joint translation and reordering models in decoding and neural models in rescoring. © 2017 Association for Computational Linguistics

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Biasing attention-based recurrent neural networks using external alignment information 2

Biasing attention-based recurrent neural networks using exte...

引用

2nd Conference on Machine Translation, WMT 2017

作者： Alkhouli, Tamer Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

ISBN: (纸本)9781945626968

This work explores extending attention-based neural models to include alignment information as input. We modify the attention component to have dependence on the current source position. The attention model is then used as a lexical model together with an additional alignment model to generate translation. The attention model is trained using external alignment information, and it is applied in decoding by performing beam search over the lexical and alignment hypotheses. The alignment model is used to score these alignment candidates. We demonstrate that the attention layer is capable of using the alignment information to improve over the baseline attention model that uses no such alignments. Our experiments are performed on two tasks: WMT 2016 English?Romanian and WMT 2017 German?English. © 2017 Association for Computational Linguistics

关键词： Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

Improved training of end-to-end attention models for speech recognition

arXiv

引用

arXiv 2018年

作者： Zeyer, Albert Irie, Kazuki Schlüter, Ralf Ney, Hermann Computer Science Department Rwth Aachen University Human Language Technology and Pattern Recognition Aachen52062 Germany AppTek United States Nnaisense Switzerland

Sequence-to-sequence attention-based models on subword units allow simple open-vocabulary end-to-end speech recognition. In this work, we show that such models can achieve competitive results on the Switchboard 300h and LibriSpeech 1000h tasks. In particular, we report the state-of-the-art word error rates (WER) of 3.54% on the dev-clean and 3.82% on the testclean evaluation subsets of LibriSpeech. We introduce a new pretraining scheme by starting with a high time reduction factor and lowering it during training, which is crucial both for convergence and final performance. In some experiments, we also use an auxiliary CTC loss function to help the convergence. In addition, we train long short-term memory (LSTM) language models on subword units. By shallow fusion, we report up to 27% relative improvements in WER over the attention baseline without a language model. Copyright © 2018, The Authors. All rights reserved.

关键词： Long short-term memory

来源：评论

学校读者我要写书评

暂无评论

Returnn: The rwth extensible training framework for universal recurrent neural networks

Returnn: The RWTH extensible training framework for universa...

引用

2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017

作者： Doetsch, Patrick Zeyer, Albert Voigtlaender, Paul Kulikov, Ilia Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University Aachen52062 Germany

ISBN: (纸本)9781509041176

In this work we release our extensible and easily configurable neural network training software. It provides a rich set of functional layers with a particular focus on efficient training of recurrent neural network topologies on multiple GPUS. The source of the software package is public and freely available for academic research purposes and can be used as a framework or as a standalone tool which supports a flexible configuration. The software allows to train state-of-the-art deep bidirectional long short-term memory (LSTM) models on both one dimensional data like speech or two dimensional data like handwritten text and was used to develop successful submission systems in several evaluation campaigns. © 2017 IEEE.

关键词： lstm multi-GPU recurrent neural networks rnn software package speech recognition

来源：评论

学校读者我要写书评

暂无评论

RADMM: RECURRENT ADAPTIVE MIXTURE MODEL WITH APPLICATIONS TO DOMAIN ROBUST language MODELING

RADMM: RECURRENT ADAPTIVE MIXTURE MODEL WITH APPLICATIONS TO...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Kazuki Irie Shankar Kumar Michael Nirschl Hank Liao Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University D-52056 Aachen Germany Google Inc. New York NY 10011 USA

ISBN: (纸本)9781538646595

We present a new architecture and a training strategy for an adaptive mixture of experts with applications to domain robust language modeling. The proposed model is designed to benefit from the scenario where the training data are available in diverse domains as is the case for YouTube speech recognition. The two core components of our model are an ensemble of parallel long short-term memory (LSTM) expert layers for each domain and another LSTM based network which generates state dependent mixture weights for combining expert LSTM states by linear interpolation. The resulting model is a recurrent adaptive mixture model (RADMM) of domain experts. We train our model on 4.4B words from YouTube speech recognition data. We report results on the YouTube speech recognition test set. Compared with a background LSTM model, we obtain up to 12% relative improvement in perplexity and an improvement in word error rate from 12.3% to 12.1% while using a lattice rescoring with strong pruning.

关键词： language modeling neural networks speech recognition mixture of experts domain adaptation modelling languages Speech recognition Neural network Professional personnel YouTube Mixture models training policy new buildings

来源：评论

学校读者我要写书评

暂无评论

MLLP-UPV and rwth aachen Spanish ASR Systems for the IberSpeech-RTVE 2018 Speech-to-Text Transcription Challenge 4

MLLP-UPV and RWTH Aachen Spanish ASR Systems for the IberSpe...

引用

4th International Conference on Advances in Speech and language Technologies for Iberian languages, IberSPEECH 2018

作者： Jorge, Javier Martínez-Villaronga, Adrià Golik, Pavel Giménez, Adrià Silvestre-Cerdà, Joan Albert Doetsch, Patrick Císcar, Vicent Andreu Ney, Hermann Juan, Alfons Sanchis, Albert Departament de Sistemes Informàtics i Computació Universitat Politècnica de València Spain Human Language Technology and Pattern Recognition RWTH Aachen University Germany Escola Tècnica Superior d'Enginyeria Informàtica Universitat Politècnica de València Spain

This paper describes the Automatic Speech recognition systems built by the MLLP research group of Universitat Politècnica de València and the HLTPR research group of rwth aachen for the IberSpeech-RTVE 2018 Speech-to-Text Transcription Challenge. We participated in both the closed and the open training conditions. The best system built for the closed condition was an hybrid BLSTM-HMM ASR system using one-pass decoding with a combination of a RNN LM and show-adapted n-gram LMs. It was trained on a set of reliable speech data extracted from the train and dev1 sets using MLLP's transLectures-UPV toolkit (TLK) and TensorFlow. This system achieved 20.0% WER on the dev2 set. For the open condition we used approx. 3800 hours of outof- domain training data from multiple sources and trained a one-pass hybrid BLSTM-HMM ASR system using open-source tools RASR and RETURNN developed at rwth aachen. This system scored 15.6% WER on the dev2 set. The highlights of these systems include robust speech data filtering for acoustic model training and show-specific language modeling. © 4th International Conference, IberSPEECH 2018.

关键词： Modeling languages

来源：评论

学校读者我要写书评

暂无评论

Local system voting feature for machine translation system combination

arXiv

引用

arXiv 2017年

作者： Freitag, Markus Peter, Jan-Thorsten Peitz, Stephan Feng, Minwei Ney, Hermann Human Language Technology Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

In this paper, we enhance the traditional confusion network system combination approach with an additional model trained by a neural network. This work is motivated by the fact that the commonly used binary system voting models only assign each input system a global weight which is responsible for the global impact of each input system on all translations. This prevents individual systems with low system weights from having influence on the system combination output, although in some situations this could be helpful. Further, words which have only been seen by one or few systems rarely have a chance of being present in the combined output. We train a local system voting model by a neural network which is based on the words themselves and the combinatorial occurrences of the different system outputs. This gives system combination the option to prefer other systems at different word positions even for the same sentence. Copyright © 2017, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Investigations on byte-level convolutional neural networks for language modeling in low resource speech recognition

Investigations on byte-level convolutional neural networks f...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Kazuki Irie Pavel Golik Ralf Schluter Hermann Ney Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University Germany

ISBN: (纸本)9781509041183

In this paper, we present an investigation on technical details of the byte-level convolutional layer which replaces the conventional linear word projection layer in the neural language model. In particular, we discuss and compare the effective filter configurations, pooling types and the use of bytes instead of characters. We carry out experiments on language packs released by the IARPA Babel project and measure the performance in terms of perplexity and word error rate. Introducing a convolutional layer consistently improves the results on all languages. Also, there is no degradation from using raw bytes instead of proper Unicode characters, even on syllabic alphabets like Amharic. In addition, we report improvements in word error rate from rescoring lattices and evaluate keyword search performance on several languages.

关键词： language modeling convolutional neural networks speech recognition keyword search modelling languages Speech recognition Key Search Error analysis language Amharic Byte Word

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：