检索结果-内蒙古大学图书馆

arXiv 2021年

作者： Bhati, Saurabhchand Villalba, Jesús Zelasko, Piotr Moro-Velazquez, Laureano Dehak, Najim Center for Language and Speech Processing Johns Hopkins University United States Human Language Technology Center of Excellence Johns Hopkins University United States

Typically, unsupervised segmentation of speech into the phone and word-like units are treated as separate tasks and are often done via different methods which do not fully leverage the inter-dependence of the two tasks. Here, we unify them and propose a technique that can jointly perform both, showing that these two tasks indeed benefit from each other. Recent attempts employ self-supervised learning, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework to model the signal structure at a higher level, e.g., phone level. A convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Experiments show that our single model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets. We discover that phone class impacts the boundary detection performance, and the boundaries between successive vowels or semivowels are the most difficult to identify. Finally, we use SCPC to extract speech features at the segment level rather than at uniformly spaced frame level (e.g., 10 ms) and produce variable rate representations that change according to the contents of the utterance. We can lower the feature extraction rate from the typical 100 Hz to as low as 14.5 Hz on average while still outperforming the MFCC features on the linear phone classification task. Copyright © 2021, The Authors. All rights reserved.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

Lhotse: A speech data representation library for the modern deep learning ecosystem

arXiv

引用

arXiv 2021年

作者： Zelasko, Piotr Povey, Daniel Trmal, Jan Khudanpur, Sanjeev Johns Hopkins University BaltimoreMD21216 United States Xiaomi China Center for Language and Speech Processing Human Language Technology Center of Excellence

speech data is notoriously difficult to work with due to a variety of codecs, lengths of recordings, and meta-data formats. We present Lhotse, a speech data representation library that draws upon lessons learned from Kaldi speech recognition toolkit and brings its concepts into the modern deep learning ecosystem. Lhotse provides a common JSON description format with corresponding Python classes and data preparation recipes for over 30 popular speech corpora. Various datasets can be easily combined together and re-purposed for different tasks. The library handles multi-channel recordings, long recordings, local and cloud storage, lazy and on-the-fly operations amongst other features. We introduce Cut and CutSet concepts, which simplify common data wrangling tasks for audio and help incorporate acoustic context of speech utterances. Finally, we show how Lhotse leverages PyTorch data API abstractions and adopts them to handle speech data for deep learning. Copyright © 2021, The Authors. All rights reserved.

关键词： Digital storage

来源：评论

学校读者我要写书评

暂无评论

USTC-NELSLIP at SemEval-2023 Task 2: Statistical Construction and Dual Adaptation of Gazetteer for Multilingual Complex NER

arXiv

引用

arXiv 2023年

作者： Ma, Jun-Yu Gu, Jia-Chen Qi, Jiajun Ling, Zhen-Hua Liu, Quan Zhao, Xiaoyi National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China China State Key Laboratory of Cognitive Intelligence iFLYTEK Research China Communication University of China China

This paper describes the system developed by the USTC-NELSLIP team for SemEval-2023 Task 2 Multilingual Complex Named Entity Recognition (MultiCoNER II). A method named Statistical Construction and Dual Adaptation of Gazetteer (SCDAG) is proposed for Multilingual Complex NER. The method first utilizes a statistics-based approach to construct a gazetteer. Secondly, the representations of gazetteer networks and language models are adapted by minimizing the KL divergence between them at both the sentence-level and entity-level. Finally, these two networks are then integrated for supervised named entity recognition (NER) training. The proposed method is applied to XLM-R with a gazetteer built from Wikidata, and shows great generalization ability across different tracks. Experimental results and detailed analysis verify the effectiveness of the proposed method. The official results show that our system ranked 1st on one track (Hindi) in this task. Copyright © 2023, The Authors. All rights reserved.

关键词： Complex networks

来源：评论

学校读者我要写书评

暂无评论

Meta Representation Learning Method for Robust Speaker Verification in Unseen Domains

Meta Representation Learning Method for Robust Speaker Verif...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Jian-Tao Zhang Yan Song Jin Li Wu Guo Hao-Yu Song Ian McLoughlin National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China iFLYTEK Research iFLYTEK Co. Ltd. Hefei China The Australian National University Australian ICT Cluster Singapore Institute of Technology Singapore

This paper presents a meta representation learning method for robust speaker verification (SV) in unseen domains. It is known that the existing embedding learning based SV systems may suffer from domain mismatch issues. To address this, we propose an episodic training procedure to compensate domain mismatch conditions at runtime. Specifically, episodes are constructed with domain balanced episodic sampling from two different domains, and a new domain alignment (DA) module is added besides the feature extractor (FE) and classifier to existing network structures. In each episodic training iteration, FE and DA modules are optimized separately with different objectives to improve the robustness of learning. Besides, a cross-domain inter-class alignment (CDICA) loss is proposed for improving the domain generalization ability. Experimental results on CNCeleb and VoxCeleb benchmarks demonstrate significant performance gains for unseen domains in SV.

关键词：

来源：评论

学校读者我要写书评

暂无评论

PQLM - MULTILINGUAL DECENTRALIZED PORTABLE QUANTUM language MODEL FOR PRIVACY PROTECTION

arXiv

引用

arXiv 2022年

作者： Li, Shuyue Stella Zhang, Xiangyu Zhou, Shu Shu, Hongchao Liang, Ruixing Liu, Hexin Garcia, Leibny Paola Center for Language and Speech Processing Johns Hopkins University United States Human Language Technology Center of Excellence Johns Hopkins University United States Department of Physics Hong Kong University of Science and Technology Hong Kong School of Electrical and Electronic Engineering Nanyang Technological University Singapore

With careful manipulation, malicious agents can reverse engineer private information encoded in pre-trained language models. Security concerns motivate the development of quantum pre-training. In this work, we propose a highly portable quantum language model (PQLM) that can easily transmit information to downstream tasks on classical machines. The framework consists of a cloud PQLM built with random Variational Quantum Classifiers (VQC) and local models for downstream applications. We demonstrate the ad hoc portability of the quantum model by extracting only the word embeddings and effectively applying them to downstream tasks on classical machines. Our PQLM exhibits comparable performance to its classical counterpart on both intrinsic evaluation (loss, perplexity) and extrinsic evaluation (multilingual sentiment analysis accuracy) metrics. We also perform ablation studies on the factors affecting PQLM performance to analyze model stability. Our work establishes a theoretical foundation for a portable quantum pre-trained language model that could be trained on private data and made available for public use with privacy protection guarantees. Copyright © 2022, The Authors. All rights reserved.

关键词： Modeling languages

来源：评论

学校读者我要写书评

暂无评论

Multimodal Tree Decoder for Table of Contents Extraction in Document Images

arXiv

引用

arXiv 2022年

作者： Hu, Pengfei Zhang, Zhenrong Zhang, Jianshu Du, Jun Wu, Jiajia National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Anhui Hefei China iFLYTEK Research China

Table of contents (ToC) extraction aims to extract headings of different levels in documents to better understand the outline of the contents, which can be widely used for document understanding and information retrieval. Existing works often use hand-crafted features and predefined rule-based functions to detect headings and resolve the hierarchical relationship between headings. Both the benchmark and research based on deep learning are still limited. Accordingly, in this paper, we first introduce a standard dataset, HierDoc, including image samples from 650 documents of scientific papers with their content labels. Then we propose a novel end-to-end model by using the multimodal tree decoder (MTD) for ToC as a benchmark for HierDoc. The MTD model is mainly composed of three parts, namely encoder, classifier, and decoder. The encoder fuses the multimodality features of vision, text, and layout information for each entity of the document. Then the classifier recognizes and selects the heading entities. Next, to parse the hierarchical relationship between the heading entities, a tree-structured decoder is designed. To evaluate the performance, both the metric of tree-edit-distance similarity (TEDS) and F1-Measure are adopted. Finally, our MTD approach achieves an average TEDS of 87.2% and an average F1-Measure of 88.1% on the test set of HierDoc. The code and dataset will be released at: https://***/Pengfei-Hu/MTD. Copyright © 2022, The Authors. All rights reserved.

关键词： Decoding

来源：评论

学校读者我要写书评

暂无评论

Multi-Class Spectral Clustering with Overlaps for Speaker Diarization

Multi-Class Spectral Clustering with Overlaps for Speaker Di...

引用

IEEE Spoken language technology Workshop

作者： Desh Raj Zili Huang Sanjeev Khudanpur Center for Language and Speech Processing The Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence The Johns Hopkins University Baltimore MD USA

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

This paper describes a method for overlap-aware speaker diarization. Given an overlap detector and a speaker embedding extractor, our method performs spectral clustering of segments informed by the output of the overlap detector. This is achieved by transforming the discrete clustering problem into a convex optimization problem which is solved by eigen-decomposition. Thereafter, we discretize the solution by alternatively using singular value decomposition and a modified version of non-maximal suppression which is constrained by the output of the overlap detector. Furthermore, we detail an HMM-DNN based overlap detector which performs frame-level classification and enforces duration constraints through HMM state transitions. Our method achieves a test diarization error rate (DER) of 24.0% on the mixed-headset setting of the AMI meeting corpus, which is a relative improvement of 15.2% over a strong agglomerative hierarchical clustering baseline, and compares favorably with other overlap-aware diarization methods. Further analysis on the LibriCSS data demonstrates the effectiveness of the proposed method in high overlap conditions.

关键词： Training Error analysis Conferences Hidden Markov models Detectors Convex functions Singular value decomposition

来源：评论

学校读者我要写书评

暂无评论

Beyond Isolated Utterances: Conversational Emotion Recognition

Beyond Isolated Utterances: Conversational Emotion Recogniti...

引用

IEEE Workshop on Automatic speech Recognition and Understanding

作者： Raghavendra Pappagari Piotr Żelasko Jesús Villalba Laureano Moro-Velazquez Najim Dehak Center for Language and Speech Processing Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD USA

ISBN: (纸本)9781665437400

speech emotion recognition is the task of recognizing the speaker's emotional state given a recording of their utterance. While most of the current approaches focus on inferring emotion from isolated utterances, we argue that this is not sufficient to achieve conversational emotion recognition (CER) which deals with recognizing emotions in conversations. In this work, we propose several approaches for CER by treating it as a sequence labeling task. We investigated transformer architecture for CER and, compared it with ResNet-34 and BiLSTM architectures in both contextual and contextless scenarios using IEMOCAP corpus. Based on the inner workings of the self-attention mechanism, we proposed DiverseCatAugment (DCA), an augmentation scheme, which improved the transformer model performance by an absolute 3.3% micro-f1 on conversations and 3.6% on isolated utterances. We further enhanced the performance by introducing an interlocutor-aware transformer model where we learn a dictionary of interlocutor index embeddings to exploit diarized conversations.

关键词： Emotion recognition Dictionaries Conferences Transformers Labeling Indexes Task analysis

来源：评论

学校读者我要写书评

暂无评论

Improving Reconstruction Loss Based Speaker Embedding in Unsupervised and Semi-Supervised Scenarios

Improving Reconstruction Loss Based Speaker Embedding in Uns...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Jaejin Cho Piotr Żelasko Jesús Villalba Najim Dehak Center for Language and Speech Processing Johns Hopkins University Baltimore MD USA Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD USA

ISBN: (纸本)9781728176055;9781728176062

Text-to-speech (TTS) models trained to minimize the spectrogram reconstruction loss can learn speaker embeddings without explicit speaker identity supervision, unlike x-vector speaker identification (SID) systems. Leveraging this way of speaker embedding learning can be useful in unsupervised or semi-supervised scenarios where non, or only some, of the training data have speaker labels. Thus, in this paper, we evaluate speaker embeddings learned by training the spectrogram prediction network under unsupervised and semi-supervised scenarios. We experimented with different data sampling strategies. The best one was sampling two different segments from the same utterance, namely A and B, where the spectrogram of B is predicted given the B phone sequence and the speaker embedding extracted from A. This method improved by 3.4% relative in EER, compared to using the same utterance for both A and B without segmenting. In the unsupervised scenario, the best speaker embedding outperformed i-vectors, the state-of-the-art unsupervised speaker embedding, in speaker verification by 12.9% relative in EER. We observed high correlation between reconstruction loss and speaker embedding quality. In the semi-supervised scenario, having more unlabeled data in training led to a better performance in speaker verification. Adding 5314 unlabeled speakers to 800 labeled speakers improved EER by 10.8 % relative.

关键词： Training Correlation Conferences Pipelines Training data speech recognition Decoding

来源：评论

学校读者我要写书评

暂无评论

Anchored Monotonic Alignment and Representation Substitution for Rare Spontaneous Behaviors in Spontaneous speech Synthesis

Anchored Monotonic Alignment and Representation Substitution...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Ning-Qian Wu Ya-Jun Hu Liping Chen Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei P.R.China iFLYTEK Research iFLYTEK Co. Ltd. China MoE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition University of Science and Technology of China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Spontaneous behaviors in speech pose significant challenges for speech synthesis. Existing research has not adequately addressed these behaviors, with most studies relying on specially recorded datasets. In contrast, real-world data more accurately reflects the natural, spontaneous speaking styles in everyday life and encompasses a wider range of spontaneous behaviors. However, such data is often of lower quality, and the distribution of spontaneous behaviors is highly imbalanced. In this study, we explore spontaneous speech synthesis using real-world data within the VITS2 framework. To overcome these challenges, we introduce two techniques: anchored monotonic alignment and spontaneous hidden representation substitution. Experimental results demonstrate that these methods enhance model alignment and improve the naturalness of the generated speech. Our proposed approach successfully addresses the challenge of synthesizing rare spontaneous behaviors and offers users flexible control over the synthesized speech.

关键词： Training Accuracy Large language models speech enhancement Signal processing Acoustics speech synthesis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：