检索结果-内蒙古大学图书馆

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Mohammad Zeineldeen Albert Zeyer Ralf Schluter Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (数字)9781509066315

ISBN: (纸本)9781509066322

Training deep neural networks is often challenging in terms of training stability. It often requires careful hyperparameter tuning or a pretraining scheme to converge. Layer normalization (LN) has shown to be a crucial ingredient in training deep encoder-decoder models. We explore various LN long short-term memory (LSTM) recurrent neural networks (RNN) variants by applying LN to different parts of the internal recurrency of LSTMs. There is no previous work that investigates this. We carry out experiments on the Switchboard 300h task for both hybrid and end-to-end ASR models and we show that LN improves the final word error rate (WER), the stability during training, allows to train even deeper models, requires less hyperparameter tuning, and works well even without pre-training. We find that applying LN to both forward and recurrent inputs globally, which we denoted by Global Joined Norm variant, gives a 10% relative improvement in WER.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Comprehensive Study of Residual CNNS for Acoustic Modeling in ASR

A Comprehensive Study of Residual CNNS for Acoustic Modeling...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Vitalii Bozheniuk Albert Zeyer Ralf Schluter Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (数字)9781509066315

ISBN: (纸本)9781509066322

Long short-term memory (LSTM) networks are the dominant architecture for large vocabulary continuous speech recognition (LVCSR) acoustic modeling due to their good performance. However, LSTMs are hard to tune and computationally expensive. To build a system with lower computational costs and which allows online streaming applications, we explore convolutional neural networks (CNN). To the best of our knowledge there is no overview on CNN hyper-parameter tuning for LVCSR in the literature, so we present our results explicitly. Apart from recognition performance, we focus on the training and evaluation speed and provide a time-efficient setup for CNNs. We faced an overfitting problem in training and solved it with data augmentation, namely SpecAugment. The system achieves results competitive with the top LSTM results. We significantly increased the speed of CNN in training and decoding approaching the speed of the offline LSTM.

关键词： acoustic modeling CNN ResNet LACE dense prediction

来源：评论

学校读者我要写书评

暂无评论

When and why is document-level context useful in neural machine translation? 4

When and why is document-level context useful in neural mach...

引用

4th Workshop on Discourse in Machine Translation, DiscoMT@EMNLP 2019

作者： Kim, Yunsu Tran, Duc Thanh Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (纸本)9781950737741

Document-level context has received lots of attention for compensating neural machine translation (NMT) of isolated sentences. However, recent advances in document-level NMT focus on sophisticated integration of the context, explaining its improvement with only a few selected examples or targeted test sets. We extensively quantify the causes of improvements by a document-level model in general test sets, clarifying the limit of the usefulness of document-level context in NMT. We show that most of the improvements are not interpretable as utilizing the context. We also show that a minimal encoding is sufficient for the context modeling and very long context is not helpful for NMT. © 2019 Association for Computational Linguistics.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

ELoPE: Fine-Grained Visual Classification with Efficient Localization, Pooling and Embedding

ELoPE: Fine-Grained Visual Classification with Efficient Loc...

引用

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Harald Hanselmann Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (数字)9781728165530

ISBN: (纸本)9781728165547

The task of fine-grained visual classification (FGVC) deals with classification problems that display a small inter-class variance such as distinguishing between different bird species or car models. State-of-the-art approaches typically tackle this problem by integrating an elaborate attention mechanism or (part-) localization method into a standard convolutional neural network (CNN). Also in this work the aim is to enhance the performance of a backbone CNN such as ResNet by including three efficient and lightweight components specifically designed for FGVC. This is achieved by using global k-max pooling, a discriminative embedding layer trained by optimizing class means and an efficient localization module that estimates bounding boxes using only class labels for training. The resulting model achieves state-of-the-art recognition accuracies on multiple FGVC benchmark datasets.

关键词： Training Task analysis Automobiles Standards Visualization Birds Testing

来源：评论

学校读者我要写书评

暂无评论

Exploring A Zero-Order Direct Hmm Based on Latent Attention for Automatic Speech recognition

Exploring A Zero-Order Direct Hmm Based on Latent Attention ...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Parnia Bahar Nikita Makarov Albert Zeyer Ralf Schluter Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany

ISBN: (数字)9781509066315

ISBN: (纸本)9781509066322

In this paper, we study a simple yet elegant latent variable attention model for automatic speech recognition (ASR) which enables an integration of attention sequence modeling into the direct hidden Markov model (HMM) concept. We use a sequence of hidden variables that establishes a mapping from output labels to input frames. Inspired by the direct HMM model, we assume a decomposition of the label sequence posterior into emission and transition probabilities using zero-order assumption and incorporate both Transformer and LSTM attention models into it. The method keeps the explicit alignment as part of the stochastic model and combines the ease of the end-to-end training of the attention model as well as an efficient and simple beam search. To study the effect of the latent model, we qualitatively analyze the alignment behavior of the different approaches. Our experiments on three ASR tasks show promising results in WER with more focused alignments in comparison to the attention models.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Interactive-Predictive Neural Multimodal Systems 9th

Interactive-Predictive Neural Multimodal Systems

引用

9th Iberian Conference on pattern recognition and Image Analysis, IbPRIA 2019

作者： Peris, Álvaro Casacuberta, Francisco Pattern Recognition and Human Language Technology Research Center Universitat Politècnica de València Valencia Spain

ISBN: (纸本)9783030313319

Despite the advances achieved by neural models in sequence to sequence learning, exploited in a variety of tasks, they still make errors. In many use cases, these are corrected by a human expert in a posterior revision process. The interactive-predictive framework aims to minimize the human effort spent on this process by considering partial corrections for iteratively refining the hypothesis. In this work, we generalize the interactive-predictive approach, typically applied in to machine translation field, to tackle other multimodal problems namely, image and video captioning. We study the application of this framework to multimodal neural sequence to sequence models. We show that, following this framework, we approximately halve the effort spent for correcting the outputs generated by the automatic systems. Moreover, we deploy our systems in a publicly accessible demonstration, that allows to better understand the behavior of the interactive-predictive framework. © 2019, Springer Nature Switzerland AG.

关键词： pattern recognition

来源：评论

学校读者我要写书评

暂无评论

Automatic Semantic Segmentation of Structural Elements related to the Spinal Cord in the Lumbar Region by using Convolutional Neural Networks

Automatic Semantic Segmentation of Structural Elements relat...

引用

International Conference on pattern recognition

作者： Jhon Jairo Saenz-Gamboa Maria de la Iglesia-Vayá Jon A. Gómez Biomedical Imaging Joint Unit Foundation for the Promotion of Health and Biomedical Research FISABIO-CIPF València Spain Pattern Recognition and Human Language Technology research center Universitat Politècnica de València València Spain

This work addresses the problem of automatically segmenting the MR images corresponding to the lumbar spine. The purpose is to detect and delimit the different structural elements like vertebrae, intervertebral discs, nerves, blood vessels, etc. This task is known as semantic segmentation. The approach proposed in this work is based on convolutional neural networks whose output is a mask where each pixel from the input image is classified into one of the possible classes. Classes were defined by radiologists and correspond to structural elements and tissues. The proposed network architectures are variants of the U-Net. Several complementary blocks were used to define the variants: spatial attention models, deep supervision and multi-kernels at input, this last block type is based on the idea of inception. Those architectures which got the best results are described in this paper, and their results are discussed. Two of the proposed architectures outperform the standard U-Net used as baseline. Those architectures which got the best results are described in this paper, and their results are discussed. Two of the proposed architectures outperform the standard U-Net used as baseline.

关键词： Image segmentation Semantics Spine Morphology Computer architecture Blood vessels Fats

来源：评论

学校读者我要写书评

暂无评论

Sample drop detection for asynchronous devices distributed in space

Sample drop detection for asynchronous devices distributed i...

引用

European Signal Processing Conference (EUSIPCO)

作者： Tina Raissi Santiago Pascual Maurizio Omologo Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany Universitat Politècnica de Catalunya Barcelona Spain Santiago Pascual is currently at Dolby Laboratories Barcelona Spain Center for Information and Communication Technology (ICT) Fondazione Bruno Kessler (FBK) Trento Italy

ISBN: (数字)9789082797053

ISBN: (纸本)9781728150017

In many applications of multi-microphone multi-device processing, the synchronization among different input channels can be affected by the lack of a common clock and isolated drops of samples. In this work, we address the issue of sample drop detection in the context of a conversational speech scenario, recorded by a set of microphones distributed in space. The goal is to design a neural-based model that given a short window in the time domain, detects whether one or more devices have been subjected to a sample drop event. The candidate time windows are selected from a set of large time intervals, possibly including a sample drop, and by using a preprocessing step. The latter is based on the application of normalized cross-correlation between signals acquired by different devices. The architecture of the neural network relies on a CNN-LSTM encoder, followed by multi-head attention. The experiments are conducted using both artificial and real data. Our proposed approach obtained F1 score of 88% on an evaluation set extracted from the CHiME-5 corpus. A comparable performance was found in a larger set of experiments conducted on a set of multi-channel artificial scenes.

关键词： Performance evaluation Array signal processing Speech recognition Synchronization Time-domain analysis Task analysis Microphones

来源：评论

学校读者我要写书评

暂无评论

ELLE: Efficient Lifelong Pre-training for Emerging Data

arXiv

引用

arXiv 2022年

作者： Qin, Yujia Zhang, Jiajie Lin, Yankai Liu, Zhiyuan Li, Peng Sun, Maosong Zhou, Jie Department of Computer Science and Technology Tsinghua University Beijing China Beijing National Research Center for Information Science and Technology China Institute for Artificial Intelligence Tsinghua University Beijing China Pattern Recognition Center WeChat AI Tencent Inc China International Innovation Center of Tsinghua University Shanghai China Beijing Academy of Artificial Intelligence China Tsinghua University China Jiangsu Collaborative Innovation Center for Language Ability Xuzhou China

Current pre-trained language models (PLM) are typically trained with static data, ignoring that in real-world scenarios, streaming data of various sources may continuously grow. This requires PLMs to integrate the information from all the sources in a lifelong manner. Although this goal could be achieved by exhaustive pre-training on all the existing data, such a process is known to be computationally expensive. To this end, we propose ELLE, aiming at efficient lifelong pretraining for emerging data. Specifically, ELLE consists of (1) function preserved model expansion, which flexibly expands an existing PLM's width and depth to improve the efficiency of knowledge acquisition;and (2) pretrained domain prompts, which disentangle the versatile knowledge learned during pretraining and stimulate the proper knowledge for downstream tasks. We experiment ELLE with streaming data from 5 domains on BERT and GPT. The results show the superiority of ELLE over various lifelong learning baselines in both pre-training efficiency and downstream performances. The codes are publicly available at https://***/thunlp/ELLE. Copyright © 2022, The Authors. All rights reserved.

关键词： Efficiency

来源：评论

学校读者我要写书评

暂无评论

Fine-Grained Visual Classification with Efficient End-to-end Localization y 202

arXiv

引用

arXiv 2020年

作者： Hanselmann, Harald Ney, Hermann Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen Germany AppTek GmbH Aachen Germany

The term fine-grained visual classification (FGVC) refers to classification tasks where the classes are very similar and the classification model needs to be able to find subtle differences to make the correct prediction. State-of-the-art approaches often include a localization step designed to help a classification network by localizing the relevant parts of the input images. However, this usually requires multiple iterations or passes through a full classification network or complex training schedules. In this work we present an efficient localization module that can be fused with a classification network in an end-to-end setup. On the one hand the module is trained by the gradient flowing back from the classification network. On the other hand, two self-supervised loss functions are introduced to increase the localization accuracy. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft and are able to achieve competitive recognition performance. Copyright © 2020, The Authors. All rights reserved.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：