检索结果-内蒙古大学图书馆

arXiv 2019年

作者： Kim, Yunsu Gao, Yingbo Ney, Hermann Human Language Technology and Pattern Recognition Group Rwth Aachen University Aachen Germany

Transfer learning or multilingual model is essential for low-resource neural machine translation (NMT), but the applicability is limited to cognate languages by sharing their vocabularies. This paper shows effective techniques to transfer a pre-trained NMT model to a new, unrelated language without shared vocabularies. We relieve the vocabulary mismatch by using cross-lingual word embedding, train a more language-agnostic encoder by injecting artificial noises, and generate synthetic data easily from the pre-training data without back-translation. Our methods do not require restructuring the vocabulary or retraining the model. We improve plain NMT transfer by up to +5.1% BLEU in five low-resource translation tasks, outperforming multilingual joint training by a large margin. We also provide extensive ablation studies on pre-trained embedding, synthetic data, vocabulary size, and parameter freezing for a better understanding of NMT transfer. Copyright © 2019, The Authors. All rights reserved.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Unsupervised adaptation of a denoising autoencoder by Bayesian Feature Enhancement for reverberant asr under mismatch conditions 40

Unsupervised adaptation of a denoising autoencoder by Bayesi...

引用

40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015

作者： Heymann, Jahn Haeb-Umbach, Reinhold Golik, Pavel Schluter, Ralf University of Paderborn Department of Communications Engineering Paderborn Germany RWTH Aachen University Human Language Technology and Pattern Recognition Computer Science Department Aachen Aachen Germany

ISBN: (纸本)9781467369978

The parametric Bayesian Feature Enhancement (BFE) and a datadriven Denoising Autoencoder (DA) both bring performance gains in severe single-channel speech recognition conditions. The first can be adjusted to different conditions by an appropriate parameter setting, while the latter needs to be trained on conditions similar to the ones expected at decoding time, making it vulnerable to a mismatch between training and test conditions. We use a DNN backend and study reverberant ASR under three types of mismatch conditions: different room reverberation times, different speaker to microphone distances and the difference between artificially reverberated data and the recordings in a reverberant environment. We show that for these mismatch conditions BFE can provide the targets for a DA. This unsupervised adaptation provides a performance gain over the direct use of BFE and even enables to compensate for the mismatch of real and simulated reverberant data. © 2015 IEEE.

关键词： deep neuronal networks denoising autoencoder feature enhancement robust speech recognition

来源：评论

学校读者我要写书评

暂无评论

Bag-of-visual-words models for adult image classification and filtering

Bag-of-visual-words models for adult image classification an...

引用

作者： Deselaers, Thomas Pimenidis, Lexi Ney, Hermann Computer Science Department Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany Security and Privacy Research RWTH Aachen University Aachen Germany

ISBN: (纸本)9781424421756

We present a method to classify images into different categories of pornographic content to create a system for filtering pornographic images from network traffic. Although different systems for this application were presented in the past, most of these systems are based on simple skin colour features and have rather poor performance. Recent advances in the image recognition field in particular for the classification of objects have shown that bag-of-visual-words-approaches are a good method for many image classification problems. The system we present here, is based on this approach, uses a task-specific visual vocabulary and is trained and evaluated on an image database of 8500 images from different categories. It is shown that it clearly outperforms earlier systems on this dataset and further evaluation on two novel web-traffic collections shows the good performance of the proposed system. © 2008 IEEE.

关键词： Image classification

来源：评论

学校读者我要写书评

暂无评论

Conditional leaving-one-out and cross-validation for discount estimation in Kneser-Ney-like extensions

Conditional leaving-one-out and cross-validation for discoun...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： J. Andrés-Ferrer M. Sundermeyer H. Ney Pattern Recognition and Human Language Technology Universidad Politécnica de Valencia Spain Human Language Technology and Pattern Recognition RWTH Aachen University Germany

The smoothing of n-gram models is a core technique in language modelling (LM). Modified Kneser-Ney (mKN) ranges among one of the best smoothing techniques. This technique discounts a fixed quantity from the observed counts in order to approximate the Turing-Good (TG) counts. Despite the TG counts optimise the leaving-one-out (L1O) criterion, the discounting parameters introduced in mKN do not. Moreover, the approximation to the TG counts for large counts is heavily simplified. In this work, both ideas are addressed: the estimation of the discounting parameters by L1O and better functional forms to approximate larger TG counts. The L1O performance is compared with cross-validation (CV) and mKN baseline in two large vocabulary tasks.

关键词： Smoothing methods Approximation methods Standards Optimization Estimation Computational modeling Training

来源：评论

学校读者我要写书评

暂无评论

Towards reinforcement learning for pivot-based neural machine translation with non-autoregressive transformer

arXiv

引用

arXiv 2021年

作者： Tokarchuk, Evgeniia Rosendahl, Jan Wang, Weiyue Petrushkov, Pavel Lancewicki, Tomer Khadivi, Shahram Ney, Hermann eBay Inc Human Language Technology and Pattern Recognition Group RWTH Aachen University Germany

Pivot-based neural machine translation (NMT) is commonly used in low-resource setups, especially for translation between non-English language pairs. It benefits from using high-resource source→pivot and pivot→target language pairs and an individual system is trained for both sub-tasks. However, these models have no connection during training, and the source→pivot model is not optimized to produce the best translation for the source→target task. In this work, we propose to train a pivot-based NMT system with the reinforcement learning (RL) approach, which has been investigated for various text generation tasks, including machine translation (MT). We utilize a non-autoregressive transformer and present an end-to-end pivot-based integrated model, enabling training on source→target data. Copyright © 2021, The Authors. All rights reserved.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Investigations on hessian-free optimization for cross-entropy training of deep neural networks

Investigations on hessian-free optimization for cross-entrop...

引用

14th Annual Conference of the International Speech Communication Association, INTERSPEECH 2013

作者： Wiesler, Simon Li, Jinyu Xue, Jian Computer Science Department Human Language Technology and Pattern Recognition RWTH Aachen University 52056 Aachen Germany Microsoft Corporation Redmond WA 98052 United States

Context-dependent deep neural network HMMs have been shown to achieve recognition accuracy superior to Gaussian mixture models in a number of recent works. Typically, neural networks are optimized with stochastic gradient descent. On large datasets, stochastic gradient descent improves quickly during the beginning of the optimization. But since it does not make use of second order information, its asymptotic convergence behavior is slow. In regions with pathological curvature, stochastic gradient descent may almost stagnate and thereby falsely indicate convergence. Another drawback of stochastic gradient descent is that it can only be parallelized within minibatches. The Hessian-free algorithm is a second order batch optimization algorithm that does not suffer from these problems. In a recent work, Hessian-free optimization has been applied to a training of deep neural networks according to a sequence criterion. In that work, improvements in accuracy and training time have been reported. In this paper, we analyze the properties of the Hessian-free optimization algorithm and investigate whether it is suited for cross-entropy training of deep neural networks as well. Copyright © 2013 ISCA.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

White-space models for offline Arabic handwriting recognition

White-space models for offline Arabic handwriting recognitio...

引用

International Conference on pattern recognition

作者： Philippe Dreuw Stephan Jonas Hermann Ney Human Language Technology and Pattern Recognition RWTH Aachen University

ISBN: (纸本)9781424421749

We propose to explicitly model white-spaces for Arabic handwriting recognition within different writing variants. Position-dependent character shapes in Arabic handwriting allow for large white-spaces between characters even within words. Here, a separate character model for white-spaces in combination with a lexicon using different writing variants and character model length adaptation is proposed. Current handwriting recognition systems model the white-spaces implicitly within the character models leading to possibly degraded models, or try to explicitly segment the Arabic words into pieces of Arabic words being prone to segmentation errors. Several white-space modeling approaches are analyzed on the well known IFN/ENIT database and outperform the best reported error rates.

关键词： White spaces Handwriting recognition Hidden Markov models Writing Plasma welding Shape Adaptation model Cities and towns Principal component analysis humans

来源：评论

学校读者我要写书评

暂无评论

Advances on the Transcription of Historical Manuscripts based on Multimodality, Interactivity and Crowdsourcing 4

Advances on the Transcription of Historical Manuscripts base...

引用

4th International Conference on Advances in Speech and language Technologies for Iberian languages, IberSPEECH 2018

作者： Granell, Emilio Martínez-Hinarejos, Carlos-D. Romero, Verónica Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camí de Vera s/n València46022 Spain

The transcription of digitalised documents is useful to ease the digital access to their contents. Natural language technologies, such as Automatic Speech recognition (ASR) for speech audio signals and Handwritten Text recognition (HTR) for text images, have become common tools for assisting transcribers, by providing a draft transcription from the digital document that they may amend. This draft is useful when it presents an error rate low enough to make the amending process more comfortable than a complete transcription from scratch. The work described in this thesis is focused on the improvement of the transcription offered by an HTR system from three scenarios: multimodality, interactivity and crowdsourcing. The image transcription can be obtained by dictating their textual contents to an ASR system. Besides, when both sources of information (image and speech) are available, a multimodal combination is possible, and this can be used to provide assistive systems with additional sources of information. Moreover, speech dictation can be used in a multimodal crowdsourcing platform, where collaborators may provide their speech by using mobile devices. Different solutions for each scenario were tested on two Spanish historical manuscripts, obtaining statistically significant improvements. © 4th International Conference, IberSPEECH 2018.

关键词： Crowdsourcing

来源：评论

学校读者我要写书评

暂无评论

Improving Transcription of Manuscripts with Multimodality and Interaction 4

Improving Transcription of Manuscripts with Multimodality an...

引用

4th International Conference on Advances in Speech and language Technologies for Iberian languages, IberSPEECH 2018

State-of-the-art Natural language recognition systems allow transcribers to speed-up the transcription of audio, video or image documents. These systems provide transcribers an initial draft transcription that can be corrected with less effort than transcribing the documents from scratch. However, even the drafts offered by the most advanced systems based on Deep Learning contain errors. Therefore, the supervision of those drafts by a human transcriber is still necessary to obtain the correct transcription. This supervision can be eased by using interactive and assistive transcription systems, where the transcriber and the automatic system cooperate in the amending process. Moreover, the interactive system can combine different sources of information in order to improve their performance, such as text line images and the dictation of their textual contents. In this paper, the performance of a multimodal interactive and assistive transcription system is evaluated on one Spanish historical manuscript. Although the quality of the draft transcriptions provided by a Handwriting Text recognition system based on Deep Learning is pretty good, the proposed interactive and assistive approach reveals an additional reduction of transcription effort. Besides, this effort reduction is increased when using speech dictations over an Automatic Speech recognition system, allowing for a faster transcription process. © 4th International Conference, IberSPEECH 2018.

关键词： human computer interaction

来源：评论

学校读者我要写书评

暂无评论

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

Controllable Factuality in Document-Grounded Dialog Systems ...

引用

2022 Findings of the Association for Computational Linguistics: EMNLP 2022

作者： Daheim, Nico Thulke, David Dugast, Christian Ney, Hermann Ubiquitous Knowledge Processing Lab Department of Computer Science Technical University of Darmstadt Germany Human Language Technology and Pattern Recognition RWTH Aachen University Germany AppTek GmbH

In this work, we present a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes' theorem. One component is a traditional ungrounded response generation model and the other component models the reconstruction of the grounding document based on the dialog context and generated response. We propose different approximate decoding schemes and evaluate our approach on multiple open-domain and task-oriented document-grounded dialog datasets. Our experiments show that the model is more factual in terms of automatic factuality metrics than the baseline model. Furthermore, we outline how introducing scaling factors between the components allows for controlling the tradeoff between factuality and fluency in the model output. Finally, we compare our approach to a recently proposed method to control factuality in grounded dialog, CTRL (Rashkin et al., 2021), and show that both approaches can be combined to achieve additional improvements. © 2022 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：