检索结果-内蒙古大学图书馆

Wake Word Detection with Streaming Transformers

学校读者我要写书评

暂无评论

Wake Word Detection with Streaming Transformers

IEEE International Conference on Acoustics, speech and Signal processing

作者： Yiming Wang Hang Lv Daniel Povey Lei Xie Sanjeev Khudanpur Center for Language and Speech Processing Johns Hopkins University Baltimore MD USA School of Computer Science Northwestern Polytechnical University Xi’an China Xiaomi Corporation Beijing China Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD USA

Modern wake word detection systems usually rely on neural networks for acoustic modeling. Transformers has recently shown superior performance over LSTM and convolutional networks in various sequence modeling tasks with their better temporal modeling power. However it is not clear whether this advantage still holds for short-range temporal modeling like wake word detection. Besides, the vanilla Transformer is not directly applicable to the task due to its non-streaming nature and the quadratic time and space complexity. In this paper we explore the performance of several variants of chunk-wise streaming Transformers tailored for wake word detection in a recently proposed LF-MMI system, including looking-ahead to the next chunk, gradient stopping, different positional embedding methods and adding same-layer dependency between chunks. Our experiments on the Mobvoi wake word dataset demonstrate that our proposed Transformer model outperforms the baseline convolution network by 25% on average in false rejection rate at the same false alarm rate with a comparable model size, while still maintaining linear complexity w.r.t. the sequence length.

关键词： Tensors Convolution System performance Conferences Neural networks Acoustics Complexity theory

Creating Multimedia Summaries Using Tweets and Videos

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Andy, Anietie Liu, Siyi Ippolito, Daphne Kriz, Reno Callison-Burch, Chris Wijaya, Derry Penn Medicine University of Pennsylvania United States Human Language Technology Center of Excellence Johns Hopkins University United States Boston University United States

While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective approach to combine social media commentary and videos to create a multimedia summary of televised events. Our approach identifies scenes from these events based on spikes of mentions of people involved in the event and automatically selects tweets and frames from the videos that occur during the time period of the spike that talk about and show the people being discussed. © 2022, CC BY.

关键词：

The JHU Multi-Microphone Multi-Speaker ASR System for the CHiME-6 Challenge

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Arora, Ashish Raj, Desh Subramanian, Aswin Shanmugam Li, Ke Ben-Yair, Bar Maciejewski, Matthew Zelasko, Piotr García, Paola Watanabe, Shinji Khudanpur, Sanjeev Center for Language and Speech Processing & Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD21218 United States

This paper summarizes the JHU team’s efforts in tracks 1 and 2 of the CHiME-6 challenge for distant multi-microphone conversational speech diarization and recognition in everyday home environments. We explore multi-array processing techniques at each stage of the pipeline, such as multi-array guided source separation (GSS) for enhancement and acoustic model training data, posterior fusion for speech activity detection, PLDA score fusion for diarization, and lattice combination for automatic speech recognition (ASR). We also report results with different acoustic model architectures, and integrate other techniques such as online multi-channel weighted prediction error (WPE) dereverberation and variational Bayes-hidden Markov model (VB-HMM) based overlap assignment to deal with reverberation and overlapping speakers, respectively. As a result of these efforts, our ASR systems achieve a word error rate of 40.5% and 67.5% on tracks 1 and 2, respectively, on the evaluation set. This is an improvement of 10.8% and 10.4% absolute, over the challenge baselines for the respective tracks. Copyright © 2020, The Authors. All rights reserved.

关键词： Microphones

MegaWika: Millions of reports and their sources across 50 diverse languages

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Barham, Samuel Weller, Orion Yuan, Michelle Murray, Kenton Yarmohammadi, Mahsa Jiang, Zhengping Vashishtha, Siddharth Martin, Alexander Liu, Anqi White, Aaron Steven Boyd-Graber, Jordan Van Durme, Benjamin Human Language Technology Center of Excellence Johns Hopkins University United States Johns Hopkins University United States University of Maryland College Park United States University of Rochester United States Amazon UMD United States

To foster the development of new models for collaborative AI-assisted report generation, we introduce MegaWika, consisting of 13 million Wikipedia articles in 50 diverse languages, along with their 71 million referenced source materials. We process this dataset for a myriad of applications, going beyond the initial Wikipedia citation extraction and web scraping of content, including translating non-English articles for cross-lingual applications and providing FrameNet parses for automated semantic analysis. MegaWika is the largest resource for sentence-level report generation and the only report generation dataset that is multilingual. We manually analyze the quality of this resource through a semantically stratified sample. Finally, we provide baseline results and trained models for crucial steps in automated report generation: cross-lingual question answering and citation retrieval. © 2023, CC BY-SA.

关键词： Semantics

Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from speech

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cho, Jaejin Villalba, Jesús Moro-Velazquez, Laureano Dehak, Najim Johns Hopkins University BaltimoreMD21218 United States The Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD21218 United States

In recent studies, self-supervised pre-trained models tend to outperform supervised pre-trained models in transfer learning. In particular, self-supervised learning of utterance-level speech representation can be used in speech applications that require discriminative representation of consistent attributes within an utterance: speaker, language, emotion, and age. Existing frame-level self-supervised speech representation, e.g., wav2vec, can be used as utterance-level representation with pooling, but the models are usually large. There are also self-supervised learning techniques to learn utterance-level representation. One of the most successful is a contrastive method, which requires negative sampling: selecting alternative samples to contrast with the current sample (anchor). However, this does not ensure that all the negative samples belong to classes different from the anchor class without labels. This paper applies a non-contrastive self-supervised method to learn utterance-level embeddings. We adapted DIstillation with NO labels (DINO) from computer vision to speech. Unlike contrastive methods, DINO does not require negative sampling. We compared DINO to x-vector trained in a supervised manner. When transferred to downstream tasks (speaker verification, speech emotion recognition, and Alzheimer's disease detection), DINO outperformed x-vector. We studied the influence of several aspects during transfer learning such as dividing the fine-tuning process into steps, chunk lengths, or augmentation. During fine-tuning, tuning the last affine layers first and then the whole network surpassed fine-tuning all at once. Using shorter chunk lengths, although they generate more diverse inputs, did not necessarily improve performance, implying speech segments at least with a specific length are required for better performance per application. Augmentation was helpful in speech emotion recognition. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

Speaker diarization using two-pass leave-one-out Gaussian PLDA clustering of DNN embeddings

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Karra, Kiran McCree, Alan Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Many modern systems for speaker diarization, such as the recently-developed VBx approach, rely on clustering of DNN speaker embeddings followed by resegmentation. Two problems with this approach are that the DNN is not directly optimized for this task, and the parameters need significant retuning for different applications. We have recently presented progress in this direction with a Leave-One-Out Gaussian PLDA (LGP) clustering algorithm and an approach to training the DNN such that embeddings directly optimize performance of this scoring method. This paper presents a new two-pass version of this system, where the second pass uses finer time resolution to significantly improve overall performance. For the Callhome corpus, we achieve the first published error rate below 4% without any task-dependent parameter tuning. We also show significant progress towards a robust single solution for multiple diarization tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

CopyPaste: An augmentation method for speech emotion recognition

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Pappagari, Raghavendra Villalba, Jesús Zelasko, Piotr Moro-Velazquez, Laureano Dehak, Najim Center for Language and Speech Processing United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Data augmentation is a widely used strategy for training robust machine learning models. It partially alleviates the problem of limited data for tasks like speech emotion recognition (SER), where collecting data is expensive and challenging. This study proposes CopyPaste, a perceptually motivated novel augmentation procedure for SER. Assuming that the presence of emotions other than neutral dictates a speaker’s overall perceived emotion in a recording, concatenation of an emotional (emotion E) and a neutral utterance can still be labeled with emotion E. We hypothesize that SER performance can be improved using these concatenated utterances in model training. To verify this, three CopyPaste schemes are tested on two deep learning models: one trained independently and another using transfer learning from an x-vector model, a speaker recognition model. We observed that all three CopyPaste schemes improve SER performance on all the three datasets considered: MSP-Podcast, Crema-D, and IEMOCAP. Additionally, CopyPaste performs better than noise augmentation and, using them together improves the SER performance further. Our experiments on noisy test sets suggested that CopyPaste is effective even in noisy test conditions. Copyright © 2020, The Authors. All rights reserved.

关键词： speech recognition

OOV Recovery with Efficient 2nd Pass Decoding and Open-vocabulary Word-level RNNLM Rescoring for Hybrid ASR

学校读者我要写书评

暂无评论

OOV Recovery with Efficient 2nd Pass Decoding and Open-vocab...

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Xiaohui Zhang Daniel Povey Sanjeev Khudanpur Facebook AI US Center for Language and Speech Processing & Human Language Technology Center of Excellence The Johns Hopkins University Baltimore MD US

ISBN: (数字)9781509066315

ISBN: (纸本)9781509066322

In this paper, we investigate out-of-vocabulary (OOV) word recovery in hybrid automatic speech recognition (ASR) systems, with emphasis on dynamic vocabulary expansion for both Weight Finite State Transducer (WFST)-based decoding and word-level RNNLM rescoring. We first describe our OOV candidate generation method based on a hybrid lexical model (HLM) with phoneme-sequence constraints. Next, we introduce a framework for efficient second pass OOV recovery with a dynamically expanded vocabulary, showing that, by calibrating OOV candidates' language model (LM) scores, it significantly improves OOV recovery and overall decoding performance compared to HLM-based first pass decoding. Finally we propose an open-vocabulary word-level recurrent neural network language model (RNNLM) re-scoring framework, making it possible to re-score ASR hypotheses containing recovered OOVs, using a single word-level RNNLM ignorant of OOVs when it was trained. By evaluating OOV recovery and overall decoding performance on Spanish/English ASR `tasks, we show the proposed OOV recovery pipeline has the potential of an efficient open-vocab word-based ASR decoding framework, with minimal extra computation versus a standard WFST based decoding and RNNLM rescoring pipeline.

关键词：

Multi-class spectral clustering with overlaps for speaker diarization

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Raj, Desh Huang, Zili Khudanpur, Sanjeev Center for Language and Speech Processing United States Human Language Technology Center of Excellence The Johns Hopkins University BaltimoreMD21218 United States

This paper describes a method for overlap-aware speaker diarization. Given an overlap detector and a speaker embedding extractor, our method performs spectral clustering of segments informed by the output of the overlap detector. This is achieved by transforming the discrete clustering problem into a convex optimization problem which is solved by eigen-decomposition. Thereafter, we discretize the solution by alternatively using singular value decomposition and a modified version of non-maximal suppression which is constrained by the output of the overlap detector. Furthermore, we detail an HMM-DNN based overlap detector which performs frame-level classification and enforces duration constraints through HMM state transitions. Our method achieves a test diarization error rate (DER) of 24.0% on the mixed-headset setting of the AMI meeting corpus, which is a relative improvement of 15.2% over a strong agglomerative hierarchical clustering baseline, and compares favorably with other overlap-aware diarization methods. Further analysis on the LibriCSS data demonstrates the effectiveness of the proposed method in high overlap conditions. © 2020, CC-BY.

关键词： Clustering algorithms