检索结果-内蒙古大学图书馆

Unsupervised speech segmentation and variable rate representation learning using segmental contrastive predictive coding

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bhati, Saurabhchand Villalba, Jesús Zelasko, Piotr Moro-Velazquez, Laureano Dehak, Najim Center for Language and Speech Processing Johns Hopkins University United States Human Language Technology Center of Excellence Johns Hopkins University United States

Typically, unsupervised segmentation of speech into the phone and word-like units are treated as separate tasks and are often done via different methods which do not fully leverage the inter-dependence of the two tasks. Here, we unify them and propose a technique that can jointly perform both, showing that these two tasks indeed benefit from each other. Recent attempts employ self-supervised learning, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework to model the signal structure at a higher level, e.g., phone level. A convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Experiments show that our single model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets. We discover that phone class impacts the boundary detection performance, and the boundaries between successive vowels or semivowels are the most difficult to identify. Finally, we use SCPC to extract speech features at the segment level rather than at uniformly spaced frame level (e.g., 10 ms) and produce variable rate representations that change according to the contents of the utterance. We can lower the feature extraction rate from the typical 100 Hz to as low as 14.5 Hz on average while still outperforming the MFCC features on the linear phone classification task. Copyright © 2021, The Authors. All rights reserved.

关键词： Signal encoding

Overview of the TREC 2023 NeuCLIR Track

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lawrie, Dawn MacAvaney, Sean Mayfield, James McNamee, Paul Oard, Douglas W. Soldaini, Luca Yang, Eugene Johns Hopkins University Human Language Technology Center of Excellence United States University of Glasgow United Kingdom University of Maryland United States Allen Institute for AI

The principal goal of the TREC Neural Cross-language Information Retrieval (NeuCLIR) track is to study the impact of neural approaches to cross-language information retrieval. The track has created four collections, large collections of Chinese, Persian, and Russian newswire and a smaller collection of Chinese scientific abstracts. The principal tasks are ranked retrieval of news in one of the three languages, using English topics. Results for a multilingual task, also with English topics but with documents from all three newswire collections, are also reported. New in this second year of the track is a pilot technical documents CLIR task for ranked retrieval of Chinese technical documents using English topics. A total of 220 runs across all tasks were submitted by six participating teams and, as baselines, by track coordinators. Task descriptions and results are presented. © 2024, CC BY.

关键词： Information retrieval

Lhotse: A speech data representation library for the modern deep learning ecosystem

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zelasko, Piotr Povey, Daniel Trmal, Jan Khudanpur, Sanjeev Johns Hopkins University BaltimoreMD21216 United States Xiaomi China Center for Language and Speech Processing Human Language Technology Center of Excellence

speech data is notoriously difficult to work with due to a variety of codecs, lengths of recordings, and meta-data formats. We present Lhotse, a speech data representation library that draws upon lessons learned from Kaldi speech recognition toolkit and brings its concepts into the modern deep learning ecosystem. Lhotse provides a common JSON description format with corresponding Python classes and data preparation recipes for over 30 popular speech corpora. Various datasets can be easily combined together and re-purposed for different tasks. The library handles multi-channel recordings, long recordings, local and cloud storage, lazy and on-the-fly operations amongst other features. We introduce Cut and CutSet concepts, which simplify common data wrangling tasks for audio and help incorporate acoustic context of speech utterances. Finally, we show how Lhotse leverages PyTorch data API abstractions and adopts them to handle speech data for deep learning. Copyright © 2021, The Authors. All rights reserved.

关键词： Digital storage

PQLM - MULTILINGUAL DECENTRALIZED PORTABLE QUANTUM language MODEL FOR PRIVACY PROTECTION

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Shuyue Stella Zhang, Xiangyu Zhou, Shu Shu, Hongchao Liang, Ruixing Liu, Hexin Garcia, Leibny Paola Center for Language and Speech Processing Johns Hopkins University United States Human Language Technology Center of Excellence Johns Hopkins University United States Department of Physics Hong Kong University of Science and Technology Hong Kong School of Electrical and Electronic Engineering Nanyang Technological University Singapore

With careful manipulation, malicious agents can reverse engineer private information encoded in pre-trained language models. Security concerns motivate the development of quantum pre-training. In this work, we propose a highly portable quantum language model (PQLM) that can easily transmit information to downstream tasks on classical machines. The framework consists of a cloud PQLM built with random Variational Quantum Classifiers (VQC) and local models for downstream applications. We demonstrate the ad hoc portability of the quantum model by extracting only the word embeddings and effectively applying them to downstream tasks on classical machines. Our PQLM exhibits comparable performance to its classical counterpart on both intrinsic evaluation (loss, perplexity) and extrinsic evaluation (multilingual sentiment analysis accuracy) metrics. We also perform ablation studies on the factors affecting PQLM performance to analyze model stability. Our work establishes a theoretical foundation for a portable quantum pre-trained language model that could be trained on private data and made available for public use with privacy protection guarantees. Copyright © 2022, The Authors. All rights reserved.

关键词： Modeling languages

Improving Reconstruction Loss Based Speaker Embedding in Unsupervised and Semi-Supervised Scenarios

学校读者我要写书评

暂无评论

Improving Reconstruction Loss Based Speaker Embedding in Uns...

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Jaejin Cho Piotr Żelasko Jesús Villalba Najim Dehak Center for Language and Speech Processing Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD USA

ISBN: (纸本)9781728176055;9781728176062

Text-to-speech (TTS) models trained to minimize the spectrogram reconstruction loss can learn speaker embeddings without explicit speaker identity supervision, unlike x-vector speaker identification (SID) systems. Leveraging this way of speaker embedding learning can be useful in unsupervised or semi-supervised scenarios where non, or only some, of the training data have speaker labels. Thus, in this paper, we evaluate speaker embeddings learned by training the spectrogram prediction network under unsupervised and semi-supervised scenarios. We experimented with different data sampling strategies. The best one was sampling two different segments from the same utterance, namely A and B, where the spectrogram of B is predicted given the B phone sequence and the speaker embedding extracted from A. This method improved by 3.4% relative in EER, compared to using the same utterance for both A and B without segmenting. In the unsupervised scenario, the best speaker embedding outperformed i-vectors, the state-of-the-art unsupervised speaker embedding, in speaker verification by 12.9% relative in EER. We observed high correlation between reconstruction loss and speaker embedding quality. In the semi-supervised scenario, having more unlabeled data in training led to a better performance in speaker verification. Adding 5314 unlabeled speakers to 800 labeled speakers improved EER by 10.8 % relative.

关键词： Training Correlation Conferences Pipelines Training data speech recognition Decoding

Beyond Isolated Utterances: Conversational Emotion Recognition

学校读者我要写书评

暂无评论

Beyond Isolated Utterances: Conversational Emotion Recogniti...

IEEE Workshop on Automatic speech Recognition and Understanding

作者： Raghavendra Pappagari Piotr Żelasko Jesús Villalba Laureano Moro-Velazquez Najim Dehak Center for Language and Speech Processing Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD USA

ISBN: (纸本)9781665437400

speech emotion recognition is the task of recognizing the speaker's emotional state given a recording of their utterance. While most of the current approaches focus on inferring emotion from isolated utterances, we argue that this is not sufficient to achieve conversational emotion recognition (CER) which deals with recognizing emotions in conversations. In this work, we propose several approaches for CER by treating it as a sequence labeling task. We investigated transformer architecture for CER and, compared it with ResNet-34 and BiLSTM architectures in both contextual and contextless scenarios using IEMOCAP corpus. Based on the inner workings of the self-attention mechanism, we proposed DiverseCatAugment (DCA), an augmentation scheme, which improved the transformer model performance by an absolute 3.3% micro-f1 on conversations and 3.6% on isolated utterances. We further enhanced the performance by introducing an interlocutor-aware transformer model where we learn a dictionary of interlocutor index embeddings to exploit diarized conversations.

关键词： Emotion recognition Dictionaries Conferences Transformers Labeling Indexes Task analysis

Multi-Class Spectral Clustering with Overlaps for Speaker Diarization

学校读者我要写书评

暂无评论

Multi-Class Spectral Clustering with Overlaps for Speaker Di...

IEEE Spoken language technology Workshop

作者： Desh Raj Zili Huang Sanjeev Khudanpur Center for Language and Speech Processing The Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence The Johns Hopkins University Baltimore MD USA

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

This paper describes a method for overlap-aware speaker diarization. Given an overlap detector and a speaker embedding extractor, our method performs spectral clustering of segments informed by the output of the overlap detector. This is achieved by transforming the discrete clustering problem into a convex optimization problem which is solved by eigen-decomposition. Thereafter, we discretize the solution by alternatively using singular value decomposition and a modified version of non-maximal suppression which is constrained by the output of the overlap detector. Furthermore, we detail an HMM-DNN based overlap detector which performs frame-level classification and enforces duration constraints through HMM state transitions. Our method achieves a test diarization error rate (DER) of 24.0% on the mixed-headset setting of the AMI meeting corpus, which is a relative improvement of 15.2% over a strong agglomerative hierarchical clustering baseline, and compares favorably with other overlap-aware diarization methods. Further analysis on the LibriCSS data demonstrates the effectiveness of the proposed method in high overlap conditions.

关键词： Training Error analysis Conferences Hidden Markov models Detectors Convex functions Singular value decomposition

Patapasco: A Python Framework for Cross-language Information Retrieval Experiments

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Costello, Cash Yang, Eugene Lawrie, Dawn Mayfield, James Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD21211 United States

While there are high-quality software frameworks for information retrieval experimentation, they do not explicitly support cross-language information retrieval (CLIR). To fill this gap, we have created Patapsco, a Python CLIR framework. This framework specifically addresses the complexity that comes with running experiments in multiple languages. Patapsco is designed to be extensible to many language pairs, to be scalable to large document collections, and to support reproducible experiments driven by a configuration file. We include Patapsco results on standard CLIR collections using multiple settings. © 2022, CC BY-NC-SA.

关键词： Python

What helps transformers recognize conversational structure? Importance of context, punctuation, and labels in dialog act recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Żelasko, Piotr Pappagari, Raghavendra Dehak, Najim Center of Language and Speech Processing Johns Hopkins University BaltimoreMD United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Dialog acts can be interpreted as the atomic units of a conversation, more fine-grained than utterances, characterized by a specific communicative function. The ability to structure a conversational transcript as a sequence of dialog acts – dialog act recognition, including the segmentation – is critical for understanding dialog. We apply two pre-trained transformer models, XLNet and Longformer, to this task in English and achieve strong results on Switchboard Dialog Act and Meeting Recorder Dialog Act corpora with dialog act segmentation error rates (DSER) of 8.4% and 14.2%. To understand the key factors affecting dialog act recognition, we perform a comparative analysis of models trained under different conditions. We find that the inclusion of a broader conversational context helps disambiguate many dialog act classes, especially those infrequent in the training data. The presence of punctuation in the transcripts has a massive effect on the models’ performance, and a detailed analysis reveals specific segmentation patterns observed in its absence. Finally, we find that the label set specificity does not affect dialog act segmentation performance. These findings have significant practical implications for spoken language understanding applications that depend heavily on a good-quality segmentation being available. Copyright © 2021, The Authors. All rights reserved.

关键词：