检索结果-内蒙古大学图书馆

On the alignment problem in multi-head attention-based neural machine translation

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Alkhouli, Tamer Bretschner, Gabriel Ney, Hermann Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University AachenD-52056 Germany

This work investigates the alignment problem in state-of-the-art multi-head attention models based on the transformer architecture. We demonstrate that alignment extraction in transformer models can be improved by augmenting an additional alignment head to the multi-head source-to-target attention component. This is used to compute sharper attention weights. We describe how to use the alignment head to achieve competitive performance. To study the effect of adding the alignment head, we simulate a dictionaryguided translation task, where the user wants to guide translation using pre-defined dictionary entries. Using the proposed approach, we achieve up to 3:8% BLEU improvement when using the dictionary, in comparison to 2:4% BLEU in the baseline case. We also propose alignment pruning to speed up decoding in alignment-based neural machine translation (ANMT), which speeds up translation by a factor of 1:8 without loss in translation performance. We carry out experiments on the shared WMT 2016 English!Romanian news task and the BOLT Chinese!English discussion forum task. Copyright © 2018, The Authors. All rights reserved.

关键词： Alignment

学校读者我要写书评

暂无评论

Deep fisher faces 28

Deep fisher faces

28th British Machine Vision Conference, BMVC 2017

作者： Hanselmann, Harald Yan, Shen Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (纸本)190172560X

Most current state-of-the-art methods for unconstrained face recognition use deep convolutional neural networks. Recently, it has been proposed to augment the typically used softmax cross-entropy loss by adding a center loss trying to minimize the distance between the face images and their class centers. In this work we further extend the center (intra-class) loss with an inter-class loss reminiscent of the popular early face recognition approach Fisherfaces. To this end we add a term that directly optimizes the distances of the class centers appearing in a batch in dependence of the input images. We evaluate the new loss on two popular databases for unconstrained face recognition, the Labeled Faces in the Wild and the Youtube Faces database. In both cases the new loss achieves competitive results. © 2017. The copyright of this document resides with its authors.

关键词： Face recognition

Complement C7 is a novel risk gene for Alzheimer's disease in Han Chinese

学校读者我要写书评

暂无评论

National Science Review 2019年第2期6卷 257-274页

作者： Deng-Feng Zhang Yu Fan Min Xu Guihong Wang Dong Wang Jin Li Li-Li Kong Hejiang Zhou Rongcan Luo Rui Bi Yong Wu Guo-Dong Li Ming Li Xiong-Jian Luo Hong-Yan Jiang Liwen Tan Chunjiu Zhong Yiru Fang Chen Zhang Nengyin Sheng Tianzi Jiang Yong-Gang Yao Key Laboratory of Animal Models and Human Disease Mechanisms of the Chinese Academy of Sciences & Yunnan Province Kunming Institute of ZoologyChinese Academy of Sciences Center for Excellence in Animal Evolution and Genetics Chinese Academy of Sciences Kunming College of Life Science University of Chinese Academy of Sciences Center for Neurodegenerative Diseases Department of Neurology Beijing Tiantan Hospital Capital Medical University Brainnetome Center Institute of AutomationChinese Academy of Sciences National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences CAS Center for Excellence in Brain Science and Intelligence Technology Chinese Academy of Sciences Department of Psychiatry the First Affiliated Hospital of Kunming Medical University Mental Health Institute of the Second Xiangya Hospital Central South University Department of Neurology Zhongshan Hospital Fudan University Division of Mood Disorders Shanghai Mental Health Center Shanghai Jiao Tong University School of Medicine State Key Laboratory of Genetic Resources and Evolution Kunming Institute of Zoology Chinese Academy of Sciences KIZ–CUHK Joint Laboratory of Bioresources and Molecular Research in Common Diseases Kunming Institute of Zoology Chinese Academy of Sciences

Alzheimer's disease is the most common neurodegenerative disease,and has a high level of genetic heritability and population *** this study,we performed the whole-exome sequencing of Han Chinese patients with familial and/or early-onset Alzheimer's disease,followed by independent validation,imaging analysis and function *** identified an exome-wide significant rare missense variant rs3792646（p.K420Q）in the C7 gene in the discovery stage(P=1.09×10-6,odds ratio=7.853)and confirmed the association in different cohorts and a combined sample(1615 cases and 2832 controls,Pcombined=2.99×10-7,odds ratio=1.930).The risk allele was associated with decreased hippocampal volume and poorer working memory performance in early adulthood,thus resulting in an earlier age of disease *** of the mutant p.K420Q disturbed cell viability,immune activation and β-amyloid *** analyses showed that the mutant p.K420Q impairs the inhibitory effect of wild type C7 on the excitatory synaptic transmission in pyramidal *** findings suggested that C7 is a novel risk gene for Alzheimer’s disease in Han Chinese.

关键词： Alzheimer’s disease whole-exome sequencing C7 neuroimaging complement system

NMT-Keras: A very flexible toolkit with a focus on interactive NMT and online learning

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Peris, Álvaro Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain

We present NMT-Keras, a flexible toolkit for training deep learning models, which puts a particularemphasis on the development of advanced applications of neural machine translation systems, such as interactive-predictive translation protocols and long-term adaptation of the translation system via continuous learning. NMT-Keras is based on an extended version of the popular Keras library, and it runs on Theano and TensorFlow. State-of-the-art neural machine translation models are deployed and used following the high-level framework provided by Keras. Given its high modularity and flexibility, it also has been extended to tackle different problems, such as image and video captioning, sentence classification and visual question answering. Copyright © 2018, The Authors. All rights reserved.

关键词： Neural machine translation

Online learning for effort reduction in interactive neural machine translation

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Peris, Álvaro Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Camino de Vera s/n Valencia46022 Spain

Neural machine translation systems require large amounts of training data and resources. Even with this, the quality of the translations may be insufficient for some users or domains. In such cases, the output of the system must be revised by a human agent. This can be done in a post-editing stage or following an interactive machine translation protocol. We explore the incremental update of neural machine translation systems during the post-editing or interactive translation processes. Such modifications aim to incorporate the new knowledge, from the edited sentences, into the translation system. Updates to the model are performed on-the-fly, as sentences are corrected, via online learning techniques. In addition, we implement a novel interactive, adaptive system, able to react to single-character interactions. This system greatly reduces the human effort required for obtaining high-quality translations. In order to stress our proposals, we conduct exhaustive experiments varying the amount and type of data available for training. Results show that online learning effectively achieves the objective of reducing the human effort required during the post-editing or the interactive machine translation stages. Moreover, these adaptive systems also perform well in scenarios with scarce resources. We show that a neural machine translation system can be rapidly adapted to a specific domain, exclusively by means of online learning techniques. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep learning

Adapting neural machine translation with parallel synthetic data 2

学校读者我要写书评

暂无评论

Adapting neural machine translation with parallel synthetic ...

2nd Conference on Machine Translation, WMT 2017

作者： Chinea-Ríos, Mara Peris, Álvaro Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València València Spain

ISBN: (纸本)9781945626968

Recent works have shown that the usage of a synthetic parallel corpus can be effectively exploited by a neural machine translation system. In this paper, we propose a new method for adapting a general neural machine translation system to a specific task, by exploiting synthetic data. The method consists in selecting, from a large monolingual pool of sentences in the source language, those instances that are more related to a given test set. Next, this selection is automatically translated and the general neural machine translation system is fine-tuned with these data. For evaluating the adaptation method, we first conducted experiments in two controlled domains, with common and well-studied corpora. Then, we evaluated our proposal on a real e-commerce task, yielding consistent improvements in terms of translation quality. © 2017 Association for Computational Linguistics

关键词： Neural machine translation

MLLP-UPV and RWTH Aachen Spanish ASR Systems for the IberSpeech-RTVE 2018 Speech-to-Text Transcription Challenge 4

学校读者我要写书评

暂无评论

MLLP-UPV and RWTH Aachen Spanish ASR Systems for the IberSpe...

4th International Conference on Advances in Speech and language Technologies for Iberian languages, IberSPEECH 2018

作者： Jorge, Javier Martínez-Villaronga, Adrià Golik, Pavel Giménez, Adrià Silvestre-Cerdà, Joan Albert Doetsch, Patrick Císcar, Vicent Andreu Ney, Hermann Juan, Alfons Sanchis, Albert Departament de Sistemes Informàtics i Computació Universitat Politècnica de València Spain Human Language Technology and Pattern Recognition RWTH Aachen University Germany Escola Tècnica Superior d'Enginyeria Informàtica Universitat Politècnica de València Spain

This paper describes the Automatic Speech recognition systems built by the MLLP research group of Universitat Politècnica de València and the HLTPR research group of RWTH Aachen for the IberSpeech-RTVE 2018 Speech-to-Text Transcription Challenge. We participated in both the closed and the open training conditions. The best system built for the closed condition was an hybrid BLSTM-HMM ASR system using one-pass decoding with a combination of a RNN LM and show-adapted n-gram LMs. It was trained on a set of reliable speech data extracted from the train and dev1 sets using MLLP's transLectures-UPV toolkit (TLK) and TensorFlow. This system achieved 20.0% WER on the dev2 set. For the open condition we used approx. 3800 hours of outof- domain training data from multiple sources and trained a one-pass hybrid BLSTM-HMM ASR system using open-source tools RASR and RETURNN developed at RWTH Aachen. This system scored 15.6% WER on the dev2 set. The highlights of these systems include robust speech data filtering for acoustic model training and show-specific language modeling. © 4th International Conference, IberSPEECH 2018.

关键词： Modeling languages

Improved training of end-to-end attention models for speech recognition

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Zeyer, Albert Irie, Kazuki Schlüter, Ralf Ney, Hermann Computer Science Department Rwth Aachen University Human Language Technology and Pattern Recognition Aachen52062 Germany AppTek United States Nnaisense Switzerland

Sequence-to-sequence attention-based models on subword units allow simple open-vocabulary end-to-end speech recognition. In this work, we show that such models can achieve competitive results on the Switchboard 300h and LibriSpeech 1000h tasks. In particular, we report the state-of-the-art word error rates (WER) of 3.54% on the dev-clean and 3.82% on the testclean evaluation subsets of LibriSpeech. We introduce a new pretraining scheme by starting with a high time reduction factor and lowering it during training, which is crucial both for convergence and final performance. In some experiments, we also use an auxiliary CTC loss function to help the convergence. In addition, we train long short-term memory (LSTM) language models on subword units. By shallow fusion, we report up to 27% relative improvements in WER over the attention baseline without a language model. Copyright © 2018, The Authors. All rights reserved.

关键词： Long short-term memory

Returnn: The RWTH extensible training framework for universal recurrent neural networks

学校读者我要写书评

暂无评论

Returnn: The RWTH extensible training framework for universa...

2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017

作者： Doetsch, Patrick Zeyer, Albert Voigtlaender, Paul Kulikov, Ilia Schlüter, Ralf Ney, Hermann Human Language Technology and Pattern Recognition Computer Science Department RWTH Aachen University Aachen52062 Germany

ISBN: (纸本)9781509041176

In this work we release our extensible and easily configurable neural network training software. It provides a rich set of functional layers with a particular focus on efficient training of recurrent neural network topologies on multiple GPUS. The source of the software package is public and freely available for academic research purposes and can be used as a framework or as a standalone tool which supports a flexible configuration. The software allows to train state-of-the-art deep bidirectional long short-term memory (LSTM) models on both one dimensional data like speech or two dimensional data like handwritten text and was used to develop successful submission systems in several evaluation campaigns. © 2017 IEEE.

关键词： lstm multi-GPU recurrent neural networks rnn software package speech recognition