检索结果-内蒙古大学图书馆

High Fidelity Speech Enhancement with Band-split RNN

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yu, Jianwei Chen, Hangting Luo, Yi Gu, Rongzhi Weng, Chao Tencent AI Lab Audio and Speech Signal Processing Oteam United States

Despite the rapid progress in speech enhancement (SE) research, improving the intelligibility and perceptual quality of desired speech in noisy environments with interfering speakers remains challenging. This paper attempts to achieve high-fidelity full-band SE and personalized SE (PSE) by modifying the recently proposed band-split RNN (BSRNN) model. To reduce the negative impact of unstable high-frequency components in full-band speech recording, we perform bi-directional and uni-directional band-level modeling to low-frequency and high-frequency subbands, respectively. For the PSE task, an additional speaker enrollment module is added to BSRNN to make use of the target speaker information for suppressing the interfering speech. Moreover, we utilize a MetricGAN discriminator (MGD) and a multi-resolution spectrogram discriminator (MRSD) to further improve the human auditory perceptual quality of the enhanced speech. Experimental results show that our system outperforms various top-ranking SE systems, achieves state-of-the-art (SOTA) SE performance on the DNS-2020 test set, and ranks among the top 3 in the DNS-2023 challenge on the PSE task. Copyright © 2022, The Authors. All rights reserved.

关键词： Speech enhancement

An NMF-based MMSE Approach for Single Channel Speech Enhancement Using Densely Connected Convolutional Network

学校读者我要写书评

暂无评论

An NMF-based MMSE Approach for Single Channel Speech Enhance...

2021 IEEE International Conference on signal processing, Communications and Computing, ICSPCC 2021

作者： Li, Xinyu Bao, Changchun Cui, Zihao Beijing University of Technology Speech and Audio Signal Processing Lab. Faculty of Information Technology Beijing100124 China

ISBN: (纸本)9781665429184

Presently, because of the development of deep learning technology, there has been increasingly more attention on state-of-The-Art masking and mapping based speech enhancement methods. However, traditional speech enhancement approaches, like minimum mean-square error (MMSE) and wiener filter (WF) have not been fully investigated. In order to the better characterize, we proposed a deep learning based MMSE approach for single-channel speech enhancement based on Non-negative Matrix Factorization (NMF). The performance of MMSE approach can be improved by a priori signal-To-noise ratio. Therefore, we utilized an NMF-based Densely Connected Convolutional Network (DenseNet) as an estimator of the a priori signal-To-noise ratio (SNR). In test stage, multiple SNR level speech from colored noise sources and real-world non-stationary noise sources were used for evaluation. As expected, our present study outperformed many previous speech enhancement methods. © 2021 IEEE.

关键词： Mean square error

audioComposer: Towards Fine-grained audio Generation with Natural Language Descriptions

学校读者我要写书评

暂无评论

AudioComposer: Towards Fine-grained Audio Generation with Na...

International Conference on Acoustics, Speech, and signal processing (ICASSP)

作者： Yuanyuan Wang Hangting Chen Dongchao Yang Zhiyong Wu Xixin Wu The Chinese University of Hong Kong Hong Kong SAR China Tencent AI Lab Audio and Speech Signal Processing Oteam China Shenzhen International Graduate School Tsinghua University Shenzhen China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Current Text-to-audio (TTA) models mainly use coarse text descriptions as inputs to generate audio, which hinders models from generating audio with fine-grained control of content and style. Some studies try to improve the granularity by incorporating additional frame-level conditions or control networks. However, this usually leads to complex system design and difficulties due to the requirement for reference frame-level conditions. To address these challenges, we propose audioComposer, a novel TTA generation framework that relies solely on natural language descriptions (NLDs) to provide both content specification and style control information. To further enhance audio generative modeling, we employ flow-based diffusion transformers with the cross-attention mechanism to incorporate text descriptions effectively into audio generation processes, which can not only simultaneously consider the content and style information in the text inputs, but also accelerate generation compared to other architectures. Furthermore, we propose a novel and comprehensive automatic data simulation pipeline to construct data with fine-grained text descriptions, which significantly alleviates the problem of data scarcity in the area. Experiments demonstrate the effectiveness of our framework using solely NLDs as inputs for content specification and style control. The generation quality and controllability surpass stateof-the-art TTA models, even with a smaller model size. 1

关键词： Natural languages Pipelines Complex networks signal processing Transformers Controllability Acoustics Speech processing Complex systems

audioComposer: Towards Fine-grained audio Generation with Natural Language Descriptions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Yuanyuan Chen, Hangting Yang, Dongchao Wu, Zhiyong Meng, Helen Wu, Xixin The Chinese University of Hong Kong Hong Kong Tencent AI Lab. Audio and Speech Signal Processing Oteam China Shenzhen International Graduate School Tsinghua University Shenzhen China

Current Text-to-audio (TTA) models mainly use coarsetext descriptions as inputs to generate audio, which hinders models from generating audio with fine-grained control of content and style. Some studies try to improve the granularity by incorporating additional frame-level conditions or control networks. However, this usually leads to complex system design and difficulties due to the requirement for reference frame-level conditions. To address these challenges, we propose audioComposer, a novel TTA generation framework that relies solely on natural language descriptions (NLDs) to provide both content specification and style control information. To further enhance audio generative modeling, we employ flow-based diffusion transformers with the cross-attention mechanism to incorporate text descriptions effectively into audio generation processes, which can not only simultaneously consider the content and style information in the text inputs, but also accelerate generation compared to other architectures. Furthermore, we propose a novel and comprehensive automatic data simulation pipeline to construct data with fine-grained text descriptions, which significantly alleviates the problem of data scarcity in the area. Experiments demonstrate the effectiveness of our framework using solely NLDs as inputs for content specification and style control. The generation quality and controllability surpass state-of-the-art TTA models, even with a smaller model size. Copyright © 2024, The Authors. All rights reserved.

关键词： Controllability

Consistent and Relevant: Rethink the Query Embedding in General Sound Separation

学校读者我要写书评

暂无评论

Consistent and Relevant: Rethink the Query Embedding in Gene...

International Conference on Acoustics, Speech, and signal processing (ICASSP)

作者： Yuanyuan Wang Hangting Chen Dongchao Yang Jianwei Yu Chao Weng Zhiyong Wu Helen Meng Shenzhen International Graduate School Tsinghua University Shenzhen China Tencent AI Lab Audio and Speech Signal Processing Oteam China The Chinese University of Hong Kong Hong Kong SAR China

The query-based audio separation usually employs specific queries to extract target sources from a mixture of audio signals. Currently, most query-based separation models need additional networks to obtain query embedding. In this way, separation model is optimized to be adapted to the distribution of query embedding. However, query embedding may exhibit mismatches with separation models due to inconsistent structures and independent information. In this paper, we present CaRE-SEP, a consistent and relevant embedding network for general sound separation to encourage a comprehensive reconsideration of query usage in audio separation. CaRE-SEP alleviates the potential mismatch between queries and separation in two aspects, including sharing network structure and sharing feature information. First, a Swin-Unet model with a shared encoder is conducted to unify query encoding and sound separation into one model, eliminating the network architecture difference and generating consistent distribution of query and separation features. Second, by initializing CaRE-SEP with a pretrained classification network and allowing gradient backpropagation, the query embedding is optimized to be relevant to the separation feature, further alleviating the feature mismatch problem. Experimental results indicate the proposed CaRE-SEP model substantially improves the performance of separation tasks. Moreover, visualizations validate the potential mismatch and how CaRE-SEP solves it.

关键词：

Exploring Receptance Weighted Key Value Model for Single-Channel Speech Enhancement

学校读者我要写书评

暂无评论

Exploring Receptance Weighted Key Value Model for Single-Cha...

IEEE International Conference on Information Communication and signal processing (ICICSP)

作者： Yuanle Li Yi Zhou Hongqing Liu School of Communications and Information Engineering Chongqing University of Posts and Telecommunications Chongqing China Chongqing Key Laboratory of Signal and Information Processing CQUPT Chongqing China Intelligent Speech and Audio Research Lab (ISARL) CQUPT Chongqing China

ISBN: (数字)9798350355895

ISBN: (纸本)9798350355901

Speech enhancement has significantly benefited from advancements in deep learning, particularly in terms of intelligibility and perceptual quality. Traditional time-frequency (TF) domain methods rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to predict TF masks or speech spectra. In this study, we introduce a novel network architecture, the Receptance Weighted Key Value (RWKV) model integrated with the Deep Complex Convolution Recurrent Network (DC-CRN), aimed at effectively training complex targets. This model combines the strengths of RNNs and Transformers, redefining the attention mechanism for speech enhancement tasks to avoid the exponential growth in computational complexity typically associated with traditional Transformer models. Through a series of experiments on the VoiceBank+Demand dataset, the RWKV-DCCRN model has demonstrated superior performance, efficiency, and scalability in handling long-range dependencies compared to traditional DCCRN models. Our implementation showcases the potential to optimize computational efficiency and performance in real-time speech enhancement applications, making it particularly suitable for environments with varying noise conditions and for devices requiring low latency processing.

关键词： Performance evaluation Training Time-frequency analysis Computational modeling Scalability Refining Speech enhancement Transformers Real-time systems Computational efficiency

CONSISTENT AND RELEVANT: RETHINK THE QUERY EMBEDDING IN GENERAL SOUND SEPARATION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yuanyuan Chen, Hangting Yang, Dongchao Yu, Jianwei Weng, Chao Wu, Zhiyong Meng, Helen Shenzhen International Graduate School Tsinghua University Shenzhen China Tencent AI Lab Audio and Speech Signal Processing Oteam China The Chinese University of Hong Kong Hong Kong

关键词： Embeddings

VIRTUAL ANALOG MODELING OF DISTORTION CIRCUITS USING NEURAL ORDINARY DIFFERENTIAL EQUATIONS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wilczek, Jan Wright, Alec Välimäki, Vesa Habets, Emanuël A.P. WolfSound Katowice Poland Acoustics Lab Dept. Signal Processing and Acoustics Aalto University Espoo Finland International Audio Laboratories Erlangen Erlangen Germany

Recent research in deep learning has shown that neural networks can learn differential equations governing dynamical systems. In this paper, we adapt this concept to Virtual Analog (VA) modeling to learn the ordinary differential equations (ODEs) governing the first-order and the second-order diode clipper. The proposed models achieve performance comparable to state-of-the-art recurrent neural networks (RNNs) albeit using fewer parameters. We show that this approach does not require oversampling and allows to increase the sampling rate after the training has completed, which results in increased accuracy. Using a sophisticated numerical solver allows to increase the accuracy at the cost of slower processing. ODEs learned this way do not require closed forms but are still physically interpretable. © 2022, CC BY.

关键词： Ordinary differential equations

UniSep: Universal Target audio Separation with Language Models at Scale

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Yuanyuan Chen, Hangting Yang, Dongchao Li, Weiqin Luo, Dan Li, Guangzhi Yang, Shan Wu, Zhiyong Meng, Helen Wu, Xixin The Chinese University of Hong Kong Hong Kong Tencent AI Lab Audio and Speech Signal Processing Oteam China Shenzhen International Graduate School Tsinghua University Shenzhen China

We propose Universal target audio Separation (UniSep), addressing the separation task on arbitrary mixtures of different types of audio. Distinguished from previous studies, UniSep is performed on unlimited source domains and unlimited source numbers. We formulate the separation task as a sequence-to-sequence problem, and a large language model (LLM) is used to model the audio sequence in the discrete latent space, leveraging the power of LLM in handling complex mixture audios with large-scale data. Moreover, a novel pre-training strategy is proposed to utilize audio-only data, which reduces the efforts of large-scale data simulation and enhances the ability of LLMs to understand the consistency and correlation of information within audio sequences. We also demonstrate the effectiveness of scaling datasets in an audio separation task: we use large-scale data (36.5k hours), including speech, music, and sound, to train a universal target audio separation model that is not limited to a specific domain. Experiments show that UniSep achieves competitive subjective and objective evaluation results compared with single-task models. Copyright © 2025, The Authors. All rights reserved.

关键词： Large datasets