检索结果-内蒙古大学图书馆

Can Automated speech Recognition Errors Provide Valuable Clues for Alzheimer’s Disease Detection?

学校读者我要写书评

暂无评论

Can Automated Speech Recognition Errors Provide Valuable Clu...

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Yin-Long Liu Rui Feng Ye-Xin Lu Jia-Xin Chen Yang Ai Jia-Hong Yuan Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei P. R. China Interdisciplinary Research Center for Linguistic Sciences University of Science and Technology of China Hefei P. R. China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Recent advances in automatic speech recognition (ASR) technology have boosted the viability of fully automated Alzheimer’s disease (AD) detection via ASR transcripts. However, there is a lack of understanding of how ASR errors affect the performance of AD detection. This paper addresses that gap. First, we fine-tune 18 ASR models on three datasets from DementiaBank, generating 36 ASR transcripts on the ADReSS dataset (18 from original and 18 from fine-tuned ASR models). We then employ two AD detection methods using either ASR or manual transcripts: fine-tuning four large language models (LLMs) and fusing LLMs with pre-trained language models (PLMs). The results show that certain ASR transcripts outperform manual transcripts, suggesting that ASR errors provide valuable clues for AD detection. Finally, we conduct an interpretability study, including linguistic and SHapley Additive exPlanations (SHAP) analyses. This study reveals that greater word distribution differences between AD and healthy control (HC) groups in ASR transcripts may be linked to these valuable clues. This paper highlights the potential of ASR as a powerful tool for developing fully automated AD detection systems.

关键词： Systematics Additives Large language models Manuals Signal processing Linguistics Acoustics Alzheimer's disease speech processing Automatic speech recognition

JOINT GENERATIVE-CONTRASTIVE REPRESENTATION LEARNING FOR ANOMALOUS SOUND DETECTION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zeng, Xiao-Min Song, Yan Zhuo, Zhu Zhou, Yu Li, Yu-Hong Xue, Hui Dai, Li-Rong McLoughlin, Ian National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China ICT Cluster Singapore Institute of Technology Singapore Alibaba Group China

In this paper, we propose a joint generative and contrastive representation learning method (GeCo) for anomalous sound detection (ASD). GeCo exploits a Predictive AutoEncoder (PAE) equipped with self-attention as a generative model to perform frame-level prediction. The output of the PAE together with original normal samples, are used for supervised contrastive representative learning in a multi-task framework. Besides cross-entropy loss between classes, contrastive loss is used to separate PAE output and original samples within each class. GeCo aims to better capture context information among frames, thanks to the self-attention mechanism for PAE model. Furthermore, GeCo combines generative and contrastive learning from which we aim to yield more effective and informative representations, compared to existing methods. Extensive experiments have been conducted on the DCASE2020 Task2 development dataset, showing that GeCo outperforms state-of-the-art generative and discriminative methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Germanium alloys

Segmental contrastive predictive coding for unsupervised word segmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bhati, Saurabhchand Villalba, Jesús Zelasko, Piotr Moro-Velazquez, Laureano Dehak, Najim Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Automatic detection of phoneme or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ self-supervised training methods, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework that can model the signal structure at a higher level e.g. at the phoneme level. In this framework, a convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Typically, phoneme and word segmentation are treated as separate tasks. We unify them and experimentally show that our single model outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and when is the right time to include the segmental loss in the learning process. Copyright © 2021, The Authors. All rights reserved.

关键词： Signal encoding

The USTC System for EEG-Music Emotion Recognition Challenge

学校读者我要写书评

暂无评论

The USTC System for EEG-Music Emotion Recognition Challenge

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Jiaxin Chen Yiming Wang Yin-Long Liu Rui Feng Jiahong Yuan Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei P. R. China Interdisciplinary Research Center for Linguistic Sciences University of Science and Technology of China Hefei P. R. China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

This paper presents the Neural Harmony team’s submission to Task 1 (Person Identification) of the ICASSP 2025 EEG-Music Emotion Recognition Challenge, which aims to identify the subject from a given EEG segment. To enhance performance, we propose a novel architecture incorporating the Multiscale ConvBlock and integrating attention mechanisms with convolutional networks. We also reprocessed the data and trained multiple models with different train-validation splits, which were ensembled during testing to further improve robustness. Our final results on the test data exceed the challenge baseline, achieving 100% accuracy in Person Identification. Additionally, unseen subjects were introduced to evaluate the model’s generalization ability, and the results confirm the model’s strong adaptability to new subjects.

关键词： Training Adaptation models Emotion recognition Accuracy Signal processing Brain modeling Robustness speech processing Identification of persons Testing

Deep feature CycleGANs: Speaker identity preserving non-parallel microphone-telephone domain adaptation for speaker verification

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Kataria, Saurabh Villalba, Jesús Żelasko, Piotr Moro-Velázquez, Laureano Dehak, Najim Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

With the increase in the availability of speech from varied domains, it is imperative to use such out-of-domain data to improve existing speech systems. Domain adaptation is a prominent pre-processing approach for this. We investigate it for adapt microphone speech to the telephone domain. Specifically, we explore CycleGAN-based unpaired translation of microphone data to improve the x-vector/speaker embedding network for Telephony Speaker Verification. We first demonstrate the efficacy of this on real challenging data and then, to improve further, we modify the CycleGAN formulation to make the adaptation task-specific. We modify CycleGAN's identity loss, cycle-consistency loss, and adversarial loss to operate in the deep feature space. Deep features of a signal are extracted from an auxiliary (speaker embedding) network and, hence, preserves speaker identity. Our 3D convolution-based Deep Feature Discriminators (DFD) show relative improvements of 5-10% in terms of equal error rate. To dive deeper, we study a challenging scenario of pooling (adapted) microphone and telephone data with data augmentations and telephone codecs. Finally, we highlight the sensitivity of CycleGAN hyper-parameters and introduce a parameter called probability of adaptation. Copyright © 2021, The Authors. All rights reserved.

关键词： speech

Earnings-21: A practical benchmark for ASR in the wild

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Del Rio, Miguel Delworth, Natalie Westerman, Ryan Huang, Michelle Bhandari, Nishchal Palakapilly, Joseph McNamara, Quinten Dong, Joshua Zelasko, Piotr Jetté, Miguel *** Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

Commonly used speech corpora inadequately challenge academic and commercial ASR systems. In particular, speech corpora lack metadata needed for detailed analysis and WER measurement. In response, we present Earnings-21, a 39-hour corpus of earnings calls containing entity-dense speech from nine different financial sectors. This corpus is intended to benchmark ASR systems in the wild with special attention towards named entity recognition. We benchmark four commercial ASR models, two internal models built with open-source tools, and an open-source Librispeech model and discuss their differences in performance on Earnings-21. Using our recently released fstalign tool, we provide a candid analysis of each model's recognition capabilities under different partitions. Our analysis finds that ASR accuracy for certain NER categories is poor, presenting a significant impediment to transcript comprehension and usage. Earnings-21 bridges academic and commercial ASR system evaluation and enables further research on entity modeling and WER on real world audio. © 2021, CC BY-SA.

关键词： speech recognition

Wav2Nas: An Exploratory Approach to Nasalance Estimation in speech

学校读者我要写书评

暂无评论

Wav2Nas: An Exploratory Approach to Nasalance Estimation in ...

International Symposium on Chinese Spoken language processing

作者： Rui Feng Yu-Ang Chen Yin-Long Liu Jia-Hong Yuan Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei Interdisciplinary Research Center for Linguistic Sciences University of Science and Technology of China Hefei

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

Nasalance, defined as the ratio of nasal energy to total acoustic energy during speech, is an important metric in speech science and clinical phonetics. Measurement of nasalance, how-ever, requires specialized equipment, which has severely limited its widespread applications. In this study, we explored methods of predicting nasalance from speech waveforms. We designed an oral-nasal separation mask with thermal flow sensors to record the airflows from the mouth and nose separately during speech production, alongside a microphone recording speech sounds. Nasalance was calculated from the oral and nasal airflows, and multilayer perceptron (MLP) models were trained to predict nasalance from speech waveforms. We compared Mel-spectrogram, Mel Frequency Cepstral Coefficients (MFCC), and Wav2vec 2.0 features as inputs to MLPs. The results demonstrated that the Wav2vec 2.0-based features have the highest Pearson Product Moment Correlation Coefficient (PPMC) of 0.7459, outperforming both the Mel-spectrogram and MFCC baselines. These findings emphasize the potential of leveraging pre-trained deep learning models such as Wav2vec 2.0 to predict nasalance directly from raw audio data, reducing reliance on expensive instruments and improving diagnostic capabilities in speech pathology. Moreover, this paper under-scores the promise of deep learning methods in advancing clinical assessment and opens up new avenues for applying compu-tational techniques to better understand and treat speech disorders.

关键词： Deep learning Pathology Nose Production Thermal sensors Predictive models Phonetics Recording Mel frequency cepstral coefficient speech processing

Beyond isolated utterances: Conversational emotion recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Pappagari, Raghavendra Zelasko, Piotr Villalba, Jesús Moro-Velazquez, Laureano Dehak, Najim Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States

speech emotion recognition is the task of recognizing the speaker's emotional state given a recording of their utterance. While most of the current approaches focus on inferring emotion from isolated utterances, we argue that this is not sufficient to achieve conversational emotion recognition (CER) which deals with recognizing emotions in conversations. In this work, we propose several approaches for CER by treating it as a sequence labeling task. We investigated transformer architecture for CER and, compared it with ResNet-34 and BiLSTM architectures in both contextual and context-less scenarios using IEMOCAP corpus. Based on the inner workings of the self-attention mechanism, we proposed DiverseCatAugment (DCA), an augmentation scheme, which improved the transformer model performance by an absolute 3.3% micro-f1 on conversations and 3.6% on isolated utterances. We further enhanced the performance by introducing an interlocutor-aware transformer model where we learn a dictionary of interlocutor index embeddings to exploit diarized conversations. Copyright © 2021, The Authors. All rights reserved.

关键词： speech recognition

Corrective Retrieval Augmented Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yan, Shi-Qi Gu, Jia-Chen Zhu, Yun Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China Department of Computer Science University of California Los Angeles United States Google DeepMind United Kingdom

Large language models (LLMs) inevitably exhibit hallucinations since the accuracy of generated texts cannot be secured solely by the parametric knowledge they encapsulate. Although retrieval-augmented generation (RAG) is a practicable complement to LLMs, it relies heavily on the relevance of retrieved documents, raising concerns about how the model behaves if retrieval goes wrong. To this end, we propose the Corrective Retrieval Augmented Generation (CRAG) to improve the robustness of generation. Specifically, a lightweight retrieval evaluator is designed to assess the overall quality of retrieved documents for a query, returning a confidence degree based on which different knowledge retrieval actions can be triggered. Since retrieval from static and limited corpora can only return suboptimal documents, large-scale web searches are utilized as an extension for augmenting the retrieval results. Besides, a decompose-then-recompose algorithm is designed for retrieved documents to selectively focus on key information and filter out irrelevant information in them. CRAG is plug-and-play and can be seamlessly coupled with various RAG-based approaches. Experiments on four datasets covering short- and long-form generation tasks show that CRAG can significantly improve the performance of RAG-based approaches. Copyright © 2024, The Authors. All rights reserved.

关键词： Structured Query language