检索结果-内蒙古大学图书馆

uniblock: Scoring and Filtering Corpus with Unicode Block Information

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Gao, Yingbo Wang, Weiyue Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

The preprocessing pipelines in Natural language Processing usually involve a step of removing sentences consisted of illegal characters. The definition of illegal characters and the specific removal strategy depend on the task, language, domain, etc, which often lead to tiresome and repetitive scripting of rules. In this paper, we introduce a simple statistical method, uniblock1, to overcome this problem. For each sentence, uniblock generates a fixed-size feature vector using Unicode block information of the characters. A Gaussian mixture model is then estimated on some clean corpus using variational inference. The learned model can then be used to score sentences and filter corpus. We present experimental results on Sentiment Analysis, language Modeling and Machine Translation, and show the simplicity and effectiveness of our method. Copyright © 2019, The Authors. All rights reserved.

关键词： Gaussian distribution

The RWTH Aachen Machine Translation System for WMT 2012 12

学校读者我要写书评

暂无评论

The RWTH Aachen Machine Translation System for WMT 2012

Workshop on Statistical Machine Translation

作者： Matthias Huck Stephan Peitz Markus Freitag Malte Nuhn Hermann Ney Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University D-52056 Aachen Germany

ISBN: (纸本)9781622765928

This paper describes the statistical machine translation (SMT) systems developed at RWTH Aachen University for the translation task of the NAACL 2012 Seventh Workshop on Statistical Machine Translation (WMT 2012). We participated in the evaluation campaign for the French-English and German-English language pairs in both translation directions. Both hierarchical and phrase-based SMT systems are applied. A number of different techniques are evaluated, including an insertion model, different lexical smoothing methods, a discriminative reordering extension for the hierarchical system, reverse translation, and system combination. By application of these methods we achieve considerable improvements over the respective baseline systems.

关键词： machine translation system machine translation Surface mount technology Hierarchical application methods Translations Translation Translation Process smoothing methods Hierarchical systems

Investigation on data adaptation techniques for neural named entity recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Tokarchuk, Evgeniia Thulke, David Wang, Weiyue Dugast, Christian Ney, Hermann Informatics Institute University of Amsterdam Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University

Data processing is an important step in various natural language processing tasks. As the commonly used datasets in named entity recognition contain only a limited number of samples, it is important to obtain additional labeled data in an efficient and reliable manner. A common practice is to utilize large monolingual unlabeled corpora. Another popular technique is to create synthetic data from the original labeled data (data augmentation). In this work, we investigate the impact of these two methods on the performance of three different named entity recognition tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Natural language processing systems

On Search Strategies for Document-Level Neural Machine Translation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Herold, Christian Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

Compared to sentence-level systems, document-level neural machine translation (NMT) models produce a more consistent output across a document and are able to better resolve ambiguities within the input. There are many works on document-level NMT, mostly focusing on modifying the model architecture or training strategy to better accommodate the additional context-input. On the other hand, in most works, the question on how to perform search with the trained model is scarcely discussed, sometimes not mentioned at all. In this work, we aim to answer the question how to best utilize a context-aware translation model in decoding. We start with the most popular document-level NMT approach and compare different decoding schemes, some from the literature and others proposed by us. In the comparison, we are using both, standard automatic metrics, as well as specific linguistic phenomena on three standard document-level translation benchmarks. We find that most commonly used decoding strategies perform similar to each other and that higher quality context information has the potential to further improve the translation. Copyright © 2023, The Authors. All rights reserved.

关键词： Neural machine translation

Improving language Model Integration for Neural Machine Translation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Herold, Christian Gao, Yingbo Zeineldeen, Mohammad Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

The integration of language models for neural machine translation has been extensively studied in the past. It has been shown that an external language model, trained on additional target-side monolingual data, can help improve translation quality. However, there has always been the assumption that the translation model also learns an implicit target-side language model during training, which interferes with the external language model at decoding time. Recently, some works on automatic speech recognition have demonstrated that, if the implicit language model is neutralized in decoding, further improvements can be gained when integrating an external language model. In this work, we transfer this concept to the task of machine translation and compare with the most prominent way of including additional monolingual data - namely back-translation. We find that accounting for the implicit language model significantly boosts the performance of language model fusion, although this approach is still outperformed by back-translation. Copyright © 2023, The Authors. All rights reserved.

关键词： Neural machine translation

Bag-of-visual-words models for adult image classification and filtering

学校读者我要写书评

暂无评论

Bag-of-visual-words models for adult image classification an...

International Conference on pattern recognition

作者： Thomas Deselaers Lexi Pimenidis Hermann Ney Human Language Technology and Pattern Recognition Security and Privacy Research Computer Science Department RWTH Aachen University Aachen Germany

ISBN: (纸本)9781424421749

We present a method to classify images into different categories of pornographic content to create a system for filtering pornographic images from network traffic. Although different systems for this application were presented in the past, most of these systems are based on simple skin colour features and have rather poor performance. Recent advances in the image recognition field in particular for the classification of objects have shown that bag-of-visual-words-approaches are a good method for many image classification problems. The system we present here, is based on this approach, uses a task-specific visual vocabulary and is trained and evaluated on an image database of 8500 images from different categories. It is shown that it clearly outperforms earlier systems on this dataset and further evaluation on two novel web-traffic collections shows the good performance of the proposed system.

关键词： Image classification Filters Vocabulary Filtering Skin Telecommunication traffic Image databases Histograms Feature extraction Iterative algorithms

Is Encoder-Decoder Redundant for Neural Machine Translation?

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gao, Yingbo Herold, Christian Yang, Zijian Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. For machine translation, despite the evolution from long short-term memory networks to Transformer networks, plus the introduction and development of attention mechanism, encoder-decoder is still the de facto neural network architecture for state-of-the-art models. While the motivation for decoding information from some hidden space is straightforward, the strict separation of the encoding and decoding steps into an encoder and a decoder in the model architecture is not necessarily a must. Compared to the task of autoregressive language modeling in the target language, machine translation simply has an additional source sentence as context. Given the fact that neural language models nowadays can already handle rather long contexts in the target language, it is natural to ask whether simply concatenating the source and target sentences and training a language model to do translation would work. In this work, we investigate the aforementioned concept for machine translation. Specifically, we experiment with bilingual translation, translation with additional target monolingual data, and multilingual translation. In all cases, this alternative approach performs on par with the baseline encoder-decoder Transformer, suggesting that an encoder-decoder architecture might be redundant for neural machine translation. Copyright © 2022, The Authors. All rights reserved.

关键词： Neural machine translation

On the alignment problem in multi-head attention-based neural machine translation

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Alkhouli, Tamer Bretschner, Gabriel Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

This work investigates the alignment problem in state-of-the-art multi-head attention models based on the transformer architecture. We demonstrate that alignment extraction in transformer models can be improved by augmenting an additional alignment head to the multi-head source-to-target attention component. This is used to compute sharper attention weights. We describe how to use the alignment head to achieve competitive performance. To study the effect of adding the alignment head, we simulate a dictionaryguided translation task, where the user wants to guide translation using pre-defined dictionary entries. Using the proposed approach, we achieve up to 3:8% BLEU improvement when using the dictionary, in comparison to 2:4% BLEU in the baseline case. We also propose alignment pruning to speed up decoding in alignment-based neural machine translation (ANMT), which speeds up translation by a factor of 1:8 without loss in translation performance. We carry out experiments on the shared WMT 2016 English!Romanian news task and the BOLT Chinese!English discussion forum task. Copyright © 2018, The Authors. All rights reserved.

关键词： Alignment

Improving Long Context Document-Level Machine Translation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Herold, Christian Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

Document-level context for neural machine translation (NMT) is crucial to improve the translation consistency and cohesion, the translation of ambiguous inputs, as well as several other linguistic phenomena. Many works have been published on the topic of document-level NMT, but most restrict the system to only local context, typically including just the one or two preceding sentences as additional information. This might be enough to resolve some ambiguous inputs, but it is probably not sufficient to capture some document-level information like the topic or style of a conversation. When increasing the context size beyond just the local context, there are two challenges: (i) the memory usage increases exponentially (ii) the translation performance starts to degrade. We argue that the widely-used attention mechanism is responsible for both issues. Therefore, we propose a constrained attention variant that focuses the attention on the most relevant parts of the sequence, while simultaneously reducing the memory consumption. For evaluation, we utilize targeted test sets in combination with novel evaluation techniques to analyze the translations in regards to specific discourse-related phenomena. We find that our approach is a good compromise between sentence-level NMT vs attending to the full context, especially in low resource scenarios. Copyright © 2023, The Authors. All rights reserved.

关键词： Neural machine translation