检索结果-内蒙古大学图书馆

Integrating Time-Frequency Domain Shallow and Deep Features for speech-EEG Match-Mismatch of Auditory Attention Decoding

引用

Journal of Shanghai Jiaotong University (Science) 2025年 1-7页

作者： Zhang, Yubang Zhu, Qiushi Xu, Qingtian Zhang, Jie National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei230026 China

Electroencephalogram (EEG) signals provide an important pathway to reflect brain activations, from which auditory attention clues of the listener can be decoded, termed as auditory attention decoding (AAD). However, existing AAD methods primarily rely on temporal or frequency features of audio and shallow features of EEG. In this work, we propose a new model fusion based AAD method with residual dilated convolution blocks, which considers both shallow and deep attention mechanisms as well as time-frequency domain features. Besides, EEG data from different utterances are mixed with the selected EEG segment for augmentation to increase the sample diversity. The effectiveness of our approach is verified by the match-mismatch task of ICASSP2024 Auditory EEG Challenge, which is a typical example of AAD. It performs much better than the baseline and state-of-the-art methods. © Shanghai Jiao Tong University 2025.

关键词： Electroencephalography

来源：评论

学校读者我要写书评

暂无评论

Joint Mixing Data Augmentation for Skeleton-Based Action Recognition

引用

ACM Transactions on Multimedia Computing, Communications and Applications 2025年第4期21卷 1-24页

作者： Xiang, Linhua Wang, Zengfu University of Science and Technology of China Hefei China National Engineering Research Centre of Speech and Language Information Processing Hefei China

Skeleton-based action recognition is beneficial for understanding human behavior in videos, and thus has received much attention in recent years as an important research area in action recognition. Current research focuses on designing more advanced algorithms to better extract spatio-temporal information from skeleton data. However, due to the small amount of data in the existing skeleton dataset and the lack of effective data augmentation methods, it is easy to lead to overfitting in model training. To address this challenge, we propose a mix-based data augmentation method, Joint Mixing Data Augmentation (JMDA), which can generally improve the effectiveness and robustness of various skeleton-based action recognition algorithms. In terms of spatial information, we introduce SpatialMix (SM), a method that projects the original 3D skeleton discrete information into a 2D space. Then, SM mixes the projected spatial information between two random samples during the training process to achieve the spatial-based mixing data augmentation. Concerning temporal information, we propose TemporalMix (TM). Leveraging the temporal continuity in skeleton data, we perform a temporal resize operation on the original skeleton data, and then merge two random samples during training to achieve the temporal-based mixed data augmentation. Additionally, we analyze the Feature Mismatch (FM) problem caused by introducing mix-based data augmentation into skeleton data. Then we propose a new data preprocessing method called Feature Alignment (FA) to effectively address this problem and improve model performance. Moreover, we propose a novel training pipeline, Joint Training Strategy (JTS), which combines multiple mix-based data augmentation methods for further improvement of model performance. Specifically, our proposed JMDA is plug-and-play and widely applicable to skeleton-based action recognition models. At the same time, the application of JMDA does not increase the model parameters and th

关键词： Musculoskeletal system

来源：评论

学校读者我要写书评

暂无评论

Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detection

Prototype based Masked Audio Model for Self-Supervised Learn...

引用

2025 IEEE International Conference on Acoustics, speech, and Signal processing, ICASSP 2025

作者： Cai, Pengfei Song, Yan Jiang, Nan Gu, Qing McLoughlin, Ian National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China China ICT Cluster Singapore Institute of Technology Singapore

ISBN: (纸本)9798350368741

A significant challenge in sound event detection (SED) is the effective utilization of unlabeled data, given the limited availability of labeled data due to high annotation costs. Semi-supervised algorithms rely on labeled data to learn from unlabeled data, and the performance is constrained by the quality and size of the former. In this paper, we introduce the Prototype based Masked Audio Model (PMAM) algorithm for self-supervised representation learning in SED, to better exploit unlabeled data. Specifically, semantically rich frame-level pseudo labels are constructed from a Gaussian mixture model (GMM) based prototypical distribution modeling. These pseudo labels supervise the leaning of a Transformer-based masked audio model, in which binary cross-entropy loss is employed instead of the widely used InfoNCE loss, to provide independent loss contributions from different prototypes, which is important in real scenarios in which multiple labels may apply to unsupervised data frames. A final stage of fine-tuning with just a small amount of labeled data yields a very high performing SED model. On like-for-like tests using the DESED task, our method achieves a PSDS1 score of 62.5%, surpassing current state-of-the-art models and demonstrating the superiority of the proposed technique. © 2025 IEEE.

关键词： masked audio model prototype self-supervised learning sound event detection

来源：评论

学校读者我要写书评

暂无评论

On the Nature and Potential of Deep Noise Suppression Embeddings

引用

Circuits, Systems, and Signal processing 2025年 1-14页

作者： McLoughlin, Ian Ding, Zhongqiang Zhang, Bowen Kurniawati, Evelyn Premkumar, A.B. Somarajan, Sasiraj Yan, Song Singapore Institute of Technology Singapore Singapore Merry Electronics Singapore Singapore Nanyang Technological University Singapore Singapore National Research Laboratory of Speech and Language Information Processing The University of Science and Technology of China Hefei China

Deep noise suppression (DNS) and AI-based speech denoising architectures learn a regression task of transforming noisy speech into clean speech. Logically, the task can be accomplished by either learning noise characteristics to identify and remove noise or by learning speech characteristics to strengthen speech with respect to noise. Architectures that employ the former approach require a good noise model, whereas the latter architectures require a strong speech model. Denoising can then be accomplished by using a noise model to mask noisy parts of the input, or by using the speech model to enhance the speech parts of the input, with both guided by appropriate training data and loss functions. Modern DNS systems are powerful and compact networks that use psychoacoustically inspired objective functions to learn their internal representations. We demonstrate that they effectively combine both approaches. This is despite having neither noise or speech labels in the training data, hence these latent representations are unsupervised. This paper explores embeddings from two recent high performance DNS architectures, to determine how they model both noise and speech across layers. Results reveal strong clustering for both speech and noise, plus significant speaker characteristic separation. This leads to a new understanding that both architectures can learn in unsupervised fashion to have speaker and noise discrimination abilities. These have strong potential to be exploited for related speaker and noise-based machine learning tasks. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025.

关键词： CRUSE DeepFilterNet DNN interpretability Embedding speech enhancement

来源：评论

学校读者我要写书评

暂无评论

The USTC System for EEG-Music Emotion Recognition Challenge

The USTC System for EEG-Music Emotion Recognition Challenge

引用

2025 IEEE International Conference on Acoustics, speech, and Signal processing, ICASSP 2025

作者： Chen, Jiaxin Wang, Yiming Liu, Yin-Long Feng, Rui Yuan, Jiahong Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China Interdisciplinary Research Center for Linguistic Sciences University of Science and Technology of China Hefei China

ISBN: (纸本)9798350368741

This paper presents the Neural Harmony team's submission to Task 1 (Person Identification) of the ICASSP 2025 EEG-Music Emotion Recognition Challenge, which aims to identify the subject from a given EEG segment. To enhance performance, we propose a novel architecture incorporating the Multiscale ConvBlock and integrating attention mechanisms with convolutional networks. We also reprocessed the data and trained multiple models with different train-validation splits, which were ensembled during testing to further improve robustness. Our final results on the test data exceed the challenge baseline, achieving 100% accuracy in Person Identification. Additionally, unseen subjects were introduced to evaluate the model's generalization ability, and the results confirm the model's strong adaptability to new subjects. © 2025 IEEE.

关键词： attention convolutional network EEG Person Identification

来源：评论

学校读者我要写书评

暂无评论

Explicit estimation of magnitude and phase spectra in parallel for high-quality speech enhancement

引用

Neural Networks 2025年 189卷 107562页

作者： Lu, Ye-Xin Ai, Yang Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China

Phase information has a significant impact on speech perceptual quality and intelligibility. However, existing speech enhancement methods encounter limitations in explicit phase estimation due to the non-structural nature and wrapping characteristics of the phase, leading to a bottleneck in enhanced speech quality. To overcome the above issue, in this paper, we proposed MP-SENet, a novel speech Enhancement Network that explicitly enhances Magnitude and Phase spectra in parallel. The proposed MP-SENet comprises a Transformer-embedded encoder–decoder architecture. The encoder aims to encode the input distorted magnitude and phase spectra into time–frequency representations, which are further fed into time–frequency Transformers for alternatively capturing time and frequency dependencies. The decoder comprises a magnitude mask decoder and a phase decoder, directly enhancing magnitude and wrapped phase spectra by incorporating a magnitude masking architecture and a phase parallel estimation architecture, respectively. Multi-level loss functions explicitly defined on the magnitude spectra, wrapped phase spectra, and short-time complex spectra are adopted to jointly train the MP-SENet model. A metric discriminator is further employed to compensate for the incomplete correlation between these losses and human auditory perception. Experimental results demonstrate that our proposed MP-SENet achieves state-of-the-art performance across multiple speech enhancement tasks, including speech denoising, dereverberation, and bandwidth extension. Compared to existing phase-aware speech enhancement methods, it further mitigates the compensation effect between the magnitude and phase by explicit phase estimation, elevating the perceptual quality of enhanced speech. Remarkably, for the speech denoising task, the proposed MP-SENet yields a PESQ of 3.60 on the VoiceBank+DEMAND dataset and 3.62 on the DNS challenge dataset. © 2025 Elsevier Ltd

关键词： speech enhancement

来源：评论

学校读者我要写书评

暂无评论

Multi-low resource languages in palm leaf manuscript recognition: Syllable-based augmentation and error analysis

引用

Pattern Recognition Letters 2025年 195卷 8-15页

作者： Thuon, Nimol Du, Jun Theang, Panhapin Thuon, Ranysakol National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei Anhui China School of Public Affairs University of Science and Technology of China Hefei Anhui China Research Innovation Department One to Many Cambodia Phnom Penh Cambodia Department of Geological Engineering Universitas Gadjah Mada Yogyakarta Indonesia

Recognizing text from palm leaf manuscripts in low-resource, non-Latin languages like Balinese, Khmer, and Sundanese poses significant challenges due to limited annotated data and complex structures. Unlike modern languages, these ancient scripts exhibit unique linguistic complexities that hinder effective recognition and digital preservation. Building on the success of syllable analysis augmentation for the Khmer script, we propose a framework, PALM-SADA, for multi-script recognition. PALM-SADA integrates visual and linguistic processing using a hybrid CNN-Transformer architecture. The framework introduces syllable analysis augmentation techniques, consisting of two main components. (1) Monosyllabic synthesis generates single-syllable words by combining glyphs from isolated glyph datasets using predefined grammar forms. And (2) Polysyllabic synthesis creates longer, grammatically correct text sequences by combining monosyllabic words and isolated glyphs. To ensure linguistic integrity, grammar forms and vocabulary lists of complete words were meticulously designed and validated, preserving the linguistic characteristics of the augmented data. For recognition, PALM-SADA employs a hybrid CNN-Transformer network that enhances both feature extraction and transcription accuracy. CNN layers capture local features, while Transformer layers model global dependencies. A Transformer-based decoder further refines transcriptions by leveraging contextual relationships within the text. Experiments conducted on the ICFHR 2018 contest datasets demonstrate that PALM-SADA significantly outperforms existing methods. © 2025 Elsevier B.V.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Voice Attribute Editing With Text Prompt

引用

IEEE Transactions on Audio, speech and language processing 2025年 33卷 1641-1652页

作者： Zheng-Yan Sheng Li-Juan Liu Yang Ai Jia Pan Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China iFLYTEK Research Hefei China

Despite recent advancements in speech generation with text prompt providing control over speech style, voice attributes in synthesized speech remain elusive and challenging to control. This paper introduces a novel task: voice attribute editing with text prompt, with the goal of making relative modifications to voice attributes according to the actions described in the text prompt. To solve this task, VoxEditor, an end-to-end generative model, is proposed. In VoxEditor, addressing the insufficiency of text prompt, a residual memory (ResMem) module is designed, that efficiently maps voice attributes and these descriptors into the shared feature space. Additionally, the ResMem module is enhanced with a voice attribute degree prediction (VADP) module to align voice attributes with corresponding descriptors, addressing the imprecision of text prompt caused by non-quantitative descriptions of voice attributes. We also establish the open-source VCTK-RVA dataset, which leads the way in manual annotations detailing voice characteristic differences among different speakers. Extensive experiments demonstrate the effectiveness and generalizability of our proposed method in terms of both objective and subjective metrics.

关键词： Annotations Training speech Magnetic domains Manuals Magnetic resonance imaging Linguistics Generative adversarial networks Face recognition Electronic mail

来源：评论

学校读者我要写书评

暂无评论

Anchored Monotonic Alignment and Representation Substitution for Rare Spontaneous Behaviors in Spontaneous speech Synthesis

Anchored Monotonic Alignment and Representation Substitution...

引用

2025 IEEE International Conference on Acoustics, speech, and Signal processing, ICASSP 2025

作者： Wu, Ning-Qian Hu, Ya-Jun Chen, Liping Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China iFLYTEK Research iFLYTEK Co. Ltd. China MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China China

ISBN: (纸本)9798350368741

Spontaneous behaviors in speech pose significant challenges for speech synthesis. Existing research has not adequately addressed these behaviors, with most studies relying on specially recorded datasets. In contrast, real-world data more accurately reflects the natural, spontaneous speaking styles in everyday life and encompasses a wider range of spontaneous behaviors. However, such data is often of lower quality, and the distribution of spontaneous behaviors is highly imbalanced. In this study, we explore spontaneous speech synthesis using real-world data within the VITS2 framework. To overcome these challenges, we introduce two techniques: anchored monotonic alignment and spontaneous hidden representation substitution. Experimental results demonstrate that these methods enhance model alignment and improve the naturalness of the generated speech. Our proposed approach successfully addresses the challenge of synthesizing rare spontaneous behaviors and offers users flexible control over the synthesized speech. © 2025 IEEE.

关键词： real-world data speech synthesis spontaneous behaviors spontaneous speech synthesis VITS

来源：评论

学校读者我要写书评

暂无评论

RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation

arXiv

引用

arXiv 2025年

作者： Yan, Shi-Qi Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei China

While Retrieval-Augmented Generation (RAG) has exhibited promise in utilizing external knowledge, its generation process heavily depends on the quality and accuracy of the retrieved context. Large language models (LLMs) struggle to evaluate the correctness of non-parametric knowledge retrieved externally when it differs from internal memorization, leading to knowledge conflicts during response generation. To this end, we introduce the Retrieval Preference Optimization (RPO), a lightweight and effective alignment method to adaptively leverage multi-source knowledge based on retrieval relevance. An implicit representation of retrieval relevance is derived and incorporated into the reward model to integrate retrieval evaluation and response generation into a single model, solving the problem that previous methods necessitate the additional procedure to assess the retrieval quality. Notably, RPO is the only RAG-dedicated alignment approach that quantifies the awareness of retrieval relevance in training, overcoming mathematical obstacles. Experiments on four datasets demonstrate that RPO outperforms RAG by 4-10% in accuracy without any extra component, exhibiting its robust generalization. Copyright © 2025, The Authors. All rights reserved.

关键词： Content based retrieval

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：