检索结果-内蒙古大学图书馆

arXiv 2023年

作者： Sustek, Martin Sadhu, Samik Burget, Lukas Hermansky, Hynek Villalba, Jesus Moro-Velazquez, Laureano Dehak, Najim Faculty of Information Technology Brno University of Technology Czechia - FIT BUT. Czech Republic Center for Language and Speech Processing Johns Hopkins University CLSP JHU United States CLSP JHU FIT BUT. HLTCOE JHU United States

The recently proposed Joint Energy-based Model (JEM) interprets discriminatively trained classifier p(y|x) as an energy model, which is also trained as a generative model describing the distribution of the input observations p(x). The JEM training relies on"positive examples" (i.e. examples from the training data set) as well as on"negative examples", which are samples from the modeled distribution p(x) generated by means of Stochastic Gradient Langevin Dynamics (SGLD). Unfortunately, SGLD often fails to deliver negative samples of sufficient quality during the standard JEM training, which causes a very unbalanced contribution from the positive and negative examples when calculating gradients for JEM updates. As a consequence, the standard JEM training is quite unstable requiring careful tuning of hyper-parameters and frequent restarts when the training starts diverging. This makes it difficult to apply JEM to different neural network architectures, modalities, and tasks. In this work, we propose a training procedure that stabilizes SGLD-based JEM training (ST-JEM) by balancing the contribution from the positive and negative examples. We also propose to add an additional"regularization" term to the training objective – MI between the input observations x and output labels y – which encourages the JEM classifier to make more certain decisions about output labels. We demonstrate the effectiveness of our approach on the CIFAR10 and CIFAR100 tasks. We also consider the task of classifying phonemes in a speech signal, for which we were not able to train JEM without the proposed stabilization. We show that a convincing speech can be generated from the trained model. Alternatively, corrupted speech can be de-noised by bringing it closer to the modeled speech distribution using a few SGLD iterations. We also propose and discuss additional applications of the trained model. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

MADNet: Maximizing Addressee Deduction Expectation for Multi-Party Conversation Generation

arXiv

引用

arXiv 2023年

作者： Gu, Jia-Chen Tan, Chao-Hong Chu, Caiyuan Ling, Zhen-Hua Tao, Chongyang Liu, Quan Liu, Cong National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China iFLYTEK Research Hefei China Peking University Beijing China State Key Laboratory of Cognitive Intelligence China

Modeling multi-party conversations (MPCs) with graph neural networks has been proven effective at capturing complicated and graphical information flows. However, existing methods rely heavily on the necessary addressee labels and can only be applied to an ideal setting where each utterance must be tagged with an "@" or other equivalent addressee label. To study the scarcity of addressee labels which is a common issue in MPCs, we propose MADNet that maximizes addressee deduction expectation in heterogeneous graph neural networks for MPC generation. Given an MPC with a few addressee labels missing, existing methods fail to build a consecutively connected conversation graph, but only a few separate conversation fragments instead. To ensure message passing between these conversation fragments, four additional types of latent edges are designed to complete a fully-connected graph. Besides, to optimize the edge-type-dependent message passing for those utterances without addressee labels, an Expectation-Maximization-based method that iteratively generates silver addressee labels (E step), and optimizes the quality of generated responses (M step), is designed. Experimental results on two Ubuntu IRC channel benchmarks show that MADNet outperforms various baseline models on the task of MPC generation, especially under the more common and challenging setting where part of addressee labels are missing. Copyright © 2023, The Authors. All rights reserved.

关键词： Message passing

来源：评论

学校读者我要写书评

暂无评论

The JHU submission to VoxSRC-21: Track 3

arXiv

引用

arXiv 2021年

作者： Cho, Jaejin Villalba, Jesús Dehak, Najim Center for Language and Speech Processing Johns Hopkins University BaltimoreMD United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

This technical report describes Johns Hopkins University speaker recognition system submitted to Voxceleb Speaker Recognition Challenge 2021 Track 3: Self-supervised speaker verification (closed). Our overall training process is similar to the proposed one from the first place team in the last year’s VoxSRC2020 challenge. The main difference is a recently proposed non-contrastive self-supervised method in computer vision (CV), distillation with no labels (DINO), is used to train our initial model, which outperformed the last year’s contrastive learning based on momentum contrast (MoCo). Also, this requires only a few iterations in the iterative clustering stage, where pseudo labels for supervised embedding learning are updated based on the clusters of the embeddings generated from a model that is continually fine-tuned over iterations. In the final stage, Res2Net50 is trained on the final pseudo labels from the iterative clustering stage. This is our best submitted model to the challenge, showing 1.89, 6.50, and 6.89 in EER(%) in voxceleb1_test_o, VoxSRC-21 validation, and test trials, respectively. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

What helps transformers recognize conversational structure? Importance of context, punctuation, and labels in dialog act recognition

arXiv

引用

arXiv 2021年

作者： Żelasko, Piotr Pappagari, Raghavendra Dehak, Najim Center of Language and Speech Processing Johns Hopkins University BaltimoreMD United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Dialog acts can be interpreted as the atomic units of a conversation, more fine-grained than utterances, characterized by a specific communicative function. The ability to structure a conversational transcript as a sequence of dialog acts – dialog act recognition, including the segmentation – is critical for understanding dialog. We apply two pre-trained transformer models, XLNet and Longformer, to this task in English and achieve strong results on Switchboard Dialog Act and Meeting Recorder Dialog Act corpora with dialog act segmentation error rates (DSER) of 8.4% and 14.2%. To understand the key factors affecting dialog act recognition, we perform a comparative analysis of models trained under different conditions. We find that the inclusion of a broader conversational context helps disambiguate many dialog act classes, especially those infrequent in the training data. The presence of punctuation in the transcripts has a massive effect on the models’ performance, and a detailed analysis reveals specific segmentation patterns observed in its absence. Finally, we find that the label set specificity does not affect dialog act segmentation performance. These findings have significant practical implications for spoken language understanding applications that depend heavily on a good-quality segmentation being available. Copyright © 2021, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Robust OpenVocabulary Translation from Visual Text Representations

arXiv

引用

arXiv 2021年

作者： Salesky, Elizabeth Etter, David Post, Matt Center for Language and Speech Processing Johns Hopkins University BaltimoreMD United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Machine translation models have discrete vocabularies and commonly use subword segmentation techniques to achieve an 'open vocabulary.' This approach relies on consistent and correct underlying unicode sequences, and makes models susceptible to degradation from common types of noise and variation. Motivated by the robustness of human language processing, we propose the use of visual text representations, which dispense with a finite set of text embeddings in favor of continuous vocabularies created by processing visually rendered text with sliding windows. We show that models using visual text representations approach or match performance of traditional text models on small and larger datasets. More importantly, models with visual embeddings demonstrate significant robustness to varied types of noise, achieving e.g., 25.9 BLEU on a character permuted German-English task where subword models degrade to 1.9. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

Representation learning to classify and detect adversarial attacks against speaker and speech recognition systems

arXiv

引用

arXiv 2021年

作者： Villalba, Jesús Joshi, Sonal Zelasko, Piotr Dehak, Najim Center for Language and Speech Processing Johns Hopkins University BaltimoreMD United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Adversarial attacks have become a major threat for machine learning applications. There is a growing interest in studying these attacks in the audio domain, e.g, speech and speaker recognition;and find defenses against them. In this work, we focus on using representation learning to classify/detect attacks w.r.t. the attack algorithm, threat model or signal-toadversarial-noise ratio. We found that common attacks in the literature can be classified with accuracies as high as 90%. Also, representations trained to classify attacks against speaker identification can be used also to classify attacks against speaker verification and speech recognition. We also tested an attack verification task, where we need to decide whether two speech utterances contain the same attack. We observed that our models did not generalize well to attack algorithms not included in the attack representation model training. Motivated by this, we evaluated an unknown attack detection task. We were able to detect unknown attacks with equal error rates of about 19%, which is promising. Copyright © 2021, The Authors. All rights reserved.

关键词： speech recognition

来源：评论

学校读者我要写书评

暂无评论

Wake word detection with streaming transformers

arXiv

引用

arXiv 2021年

作者： Wang, Yiming Lv, Hang Povey, Daniel Xie, Lei Khudanpur, Sanjeev Center for Language and Speech Processing Johns Hopkins University BaltimoreMD United States Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States ASLP@NPU School of Computer Science Northwestern Polytechnical University Xi’an China Xiaomi Corporation Beijing China

Modern wake word detection systems usually rely on neural networks for acoustic modeling. Transformers has recently shown superior performance over LSTM and convolutional networks in various sequence modeling tasks with their better temporal modeling power. However it is not clear whether this advantage still holds for short-range temporal modeling like wake word detection. Besides, the vanilla Transformer is not directly applicable to the task due to its non-streaming nature and the quadratic time and space complexity. In this paper we explore the performance of several variants of chunk-wise streaming Transformers tailored for wake word detection in a recently proposed LF-MMI system, including looking-ahead to the next chunk, gradient stopping, different positional embedding methods and adding same-layer dependency between chunks. Our experiments on the Mobvoi wake word dataset demonstrate that our proposed Transformer model outperforms the baseline convolution network by 25% on average in false rejection rate at the same false alarm rate with a comparable model size, while still maintaining linear complexity w.r.t. the sequence length. Copyright © 2021, The Authors. All rights reserved.

关键词： Wakes

来源：评论

学校读者我要写书评

暂无评论

A Parallelizable Lattice Rescoring Strategy with Neural language Models

A Parallelizable Lattice Rescoring Strategy with Neural Lang...

引用

IEEE International Conference on Acoustics, speech and Signal processing

作者： Ke Li Daniel Povey Sanjeev Khudanpur Center for Language and Speech Processing The Johns Hopkins University Baltimore MD USA Xiaomi Corp Beijing China Human Language Technology Center of Excellence The Johns Hopkins University Baltimore MD USA

ISBN: (纸本)9781728176055;9781728176062

This paper proposes a parallel computation strategy and a posterior-based lattice expansion algorithm for efficient lattice rescoring with neural language models (LMs) for automatic speech recognition. First, lattices from first-pass decoding are expanded by the proposed posterior-based lattice expansion algorithm. Second, each expanded lattice is converted into a minimal list of hypotheses that covers every arc. Each hypothesis is constrained to be the best path for at least one arc it includes. For each lattice, the neural LM scores of the minimal list are computed in parallel and are then integrated back to the lattice in the rescoring stage. Experiments on the Switchboard dataset show that the proposed rescoring strategy obtains comparable recognition performance and generates more compact lattices than a competitive baseline method. Furthermore, the parallel rescoring method offers more flexibility by simplifying the integration of PyTorch-trained neural LMs for lattice rescoring with Kaldi.

关键词： Computational modeling Conferences Lattices Signal processing algorithms Switches Signal processing Decoding

来源：评论

学校读者我要写书评

暂无评论

Reformulating DOVER-Lap label mapping as a graph partitioning problem

arXiv

引用

arXiv 2021年

作者： Raj, Desh Khudanpur, Sanjeev Center for Language and Speech Processing Johns Hopkins University BaltimoreMD21218 United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD21218 United States

We recently proposed DOVER-Lap, a method for combining overlap-aware speaker diarization system outputs. DOVER-Lap improved upon its predecessor DOVER by using a label mapping method based on globally-informed greedy search. In this paper, we analyze this label mapping in the framework of a maximum orthogonal graph partitioning problem, and present three inferences. First, we show that DOVER-Lap label mapping is exponential in the input size, which poses a challenge when combining a large number of hypotheses. We then revisit the DOVER label mapping algorithm and propose a modification which performs similar to DOVER-Lap while being computationally tractable. We also derive an approximation bound for the algorithm in terms of the maximum number of hypotheses speakers. Finally, we describe a randomized local search algorithm which provides a near-optimal (1 - Ε)approximate solution to the problem with high probability. We empirically demonstrate the effectiveness of our methods on the AMI meeting corpus. Our code is publicly available: https://***/desh2608/dover-lap. © 2021, CC BY.

关键词： Approximation algorithms

来源：评论

学校读者我要写书评

暂无评论

DOVER-Lap: A Method for Combining Overlap-Aware Diarization Outputs

DOVER-Lap: A Method for Combining Overlap-Aware Diarization ...

引用

IEEE Spoken language technology Workshop

作者： Desh Raj Leibny Paola Garcia-Perera Zili Huang Shinji Watanabe Daniel Povey Andreas Stolcke Sanjeev Khudanpur Center for Language and Speech Processing The Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence The Johns Hopkins University Baltimore MD USA Xiaomi Corp. Beijing China Amazon Alexa Speech Sunnyvale CA USA

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

Several advances have been made recently towards handling overlapping speech for speaker diarization. Since speech and natural language tasks often benefit from ensemble techniques, we propose an algorithm for combining outputs from such diarization systems through majority voting. Our method, DOVER-Lap, is inspired from the recently proposed DOVER algorithm, but is designed to handle overlapping segments in diarization outputs. We also modify the pair-wise incremental label mapping strategy used in DOVER, and propose an approximation algorithm based on weighted k-partite graph matching, which performs this mapping using a global cost tensor. We demonstrate the strength of our method by combining outputs from diverse systems - clustering-based, region proposal networks, and target-speaker voice activity detection - on AMI and LibriCSS datasets, where it consistently outperforms the single best system. Additionally, we show that DOVER-Lap can be used for late fusion in multichannel diarization, and compares favorably with early fusion methods like beamforming.

关键词： Voice activity detection Tensors Natural languages Diversity reception Approximation algorithms Proposals Task analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：