检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Ai, Yang Jiang, Xiao-Hang Lu, Ye-Xin Du, Hui-Peng Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei230027 China

This paper introduces a novel neural audio codec targeting high waveform sampling rates and low bitrates named APCodec, which seamlessly integrates the strengths of parametric codecs and waveform codecs. The APCodec revolutionizes the process of audio encoding and decoding by concurrently handling the amplitude and phase spectra as audio parametric characteristics like parametric codecs. It is composed of an encoder and a decoder with the modified ConvNeXt v2 network as the backbone, connected by a quantizer based on the residual vector quantization (RVQ) mechanism. The encoder compresses the audio amplitude and phase spectra in parallel, amalgamating them into a continuous latent code at a reduced temporal resolution. This code is subsequently quantized by the quantizer. Ultimately, the decoder reconstructs the audio amplitude and phase spectra in parallel, and the decoded waveform is obtained by inverse short-time Fourier transform. To ensure the fidelity of decoded audio like waveform codecs, spectral-level loss, quantization loss, and generative adversarial network (GAN) based loss are collectively employed for training the APCodec. To support low-latency streamable inference, we employ feed-forward layers and causal deconvolutional layers in APCodec, incorporating a knowledge distillation training strategy to enhance the quality of decoded audio. Experimental results confirm that our proposed APCodec can encode 48 kHz audio at bitrate of just 6 kbps, with no significant degradation in the quality of the decoded audio. At the same bitrate, our proposed APCodec also demonstrates superior decoded audio quality and faster generation speed compared to well-known codecs, such as Encodec, AudioDec and DAC. Copyright © 2024, The Authors. All rights reserved.

关键词： Vector quantization

来源：评论

学校读者我要写书评

暂无评论

ADAPTING SELF-SUPERVISED MODELS TO MULTI-TALKER speech RECOGNITION USING SPEAKER EMBEDDINGS

arXiv

引用

arXiv 2022年

作者： Huang, Zili Raj, Desh García, Paola Khudanpur, Sanjeev Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University Baltimore United States

Self-supervised learning (SSL) methods which learn representations of data without explicit supervision have gained popularity in speech-processing tasks, particularly for single-talker applications. However, these models often have degraded performance for multi-talker scenarios — possibly due to the domain mismatch — which severely limits their use for such applications. In this paper, we investigate the adaptation of upstream SSL models to the multi-talker automatic speech recognition (ASR) task under two conditions. First, when segmented utterances are given, we show that adding a target speaker extraction (TSE) module based on enrollment embeddings is complementary to mixture-aware pre-training. Second, for unsegmented mixtures, we propose a novel joint speaker modeling (JSM) approach, which aggregates information from all speakers in the mixture through their embeddings. With controlled experiments on Libri2Mix, we show that using speaker embeddings provides relative WER improvements of 9.1% and 42.1% over strong baselines for the segmented and unsegmented cases, respectively. We also demonstrate the effectiveness of our models for real conversational mixtures through experiments on the AMI dataset. © 2022, CC BY.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

Towards High-Quality and Efficient speech Bandwidth Extension with Parallel Amplitude and Phase Prediction

arXiv

引用

arXiv 2024年

作者： Lu, Ye-Xin Ai, Yang Du, Hui-Peng Ling, Zhen-Hua National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei230027 China

speech bandwidth extension (BWE) refers to widening the frequency bandwidth range of speech signals, enhancing the speech quality towards brighter and fuller. This paper proposes a generative adversarial network (GAN) based BWE model with parallel prediction of Amplitude and Phase spectra, named AP-BWE, which achieves both high-quality and efficient wideband speech waveform generation. The proposed AP-BWE generator is entirely based on convolutional neural networks (CNNs). It features a dual-stream architecture with mutual interaction, where the amplitude stream and the phase stream communicate with each other and respectively extend the high-frequency components from the source narrowband amplitude and phase spectra. To improve the naturalness of the extended speech signals, we employ a multi-period discriminator at the waveform level and design a pair of multi-resolution amplitude and phase discriminators at the spectral level, respectively. Experimental results demonstrate that our proposed AP-BWE achieves state-of-the-art performance in terms of speech quality for BWE tasks targeting sampling rates of both 16 kHz and 48 kHz. In terms of generation efficiency, due to the all-convolutional architecture and all-frame-level operations, the proposed AP-BWE can generate 48 kHz waveform samples 292.3 times faster than real-time on a single RTX 4090 GPU and 18.1 times faster than real-time on a single CPU. Notably, to our knowledge, AP-BWE is the first to achieve the direct extension of the high-frequency phase spectrum, which is beneficial for improving the effectiveness of existing BWE methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Discriminators

来源：评论

学校读者我要写书评

暂无评论

Aligning Noisy-Clean speech Pairs at Feature and Embedding Levels for Learning Noise-Invariant Speaker Representations

Aligning Noisy-Clean Speech Pairs at Feature and Embedding L...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Zuoliang Li Yang Ai Jie Zhang Shengyu Peng Yu Guan Bin Gu Wu Guo The National Engineering Research Center for Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China (USTC) Hefei China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

In this paper, we propose a noise-invariant speaker representation learning (SRL) approach by aligning noisy-clean speech pairs at both the feature and embedding levels for model training. Specifically, we first construct noisy-clean pairs using data augmentation during training. The noisy features are then processed by a Conformer-based enhancement module. The feature-level alignment is achieved by minimizing the mean squared error between the enhanced and original clean data. At the embedding level, we introduce a supervised contrastive learning loss with noise-adaptive margin to simultaneously enhance the intra-speaker compactness and the inter-speaker separability and better adapt different noise levels, in combination with the Barlow Twins self-supervised loss to align the noisy-clean data pairs and reduce noise redundancy in the embedding space. Finally, these loss components are integrated with conventional classification loss to train the SRL network. Experimental results on various VoxCeleb1 test sets synthesized with noise sources demonstrate the effectiveness of the proposed method.

关键词： Training Representation learning Noise Redundancy Contrastive learning Feature extraction Data augmentation Noise measurement speech processing Noise level

来源：评论

学校读者我要写书评

暂无评论

Recursive Feature Learning from Pre-Trained Models for Spoofing speech Detection

Recursive Feature Learning from Pre-Trained Models for Spoof...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Yu Guan Yang Ai Zuoliang Li Shengyu Peng Wu Guo National Engineering Research Center for Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China (USTC) Hefei China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

It was recently revealed that using features extracted from pre-trained models can achieve much better performance than using conventional hand-crafted acoustic features for spoofing speech detection. In this paper, we therefore enhance the features from pre-trained model based on recursive learning. Specifically, we modify the pre-trained model by feeding the features from the topmost transformer layer to bottom layers recursively, and the obtained recursive features from the bottom layers are fused with that from topmost layer. The fused features are then fed into the backend classifiers. Experiments are carried out on two benchmark datasets (i.e., ASVspoof 2019 LA and ASVspoof 2021 LA), which show the superiority of the proposed method over state-of-the-art systems.

关键词： Voice activity detection Representation learning Linear regression Signal processing Benchmark testing Feature extraction Transformers Excavation Acoustics

来源：评论

学校读者我要写书评

暂无评论

CASC-XVC: Zero-Shot Cross-Lingual Voice Conversion with Content Accordant and Speaker Contrastive Losses

CASC-XVC: Zero-Shot Cross-Lingual Voice Conversion with Cont...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Han-Jie Guo Hui-Peng Du Zheng-Yan Sheng Li-Ping Chen Yang Ai Zhen-Hua Ling National Engineering Research Center of Speech and Language Information Processing University of Science and Technology of China Hefei P.R.China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Cross-lingual voice conversion (XVC) is a technology that modifies speaker identity while preserving linguistic content in scenarios where the source and target speakers use different languages. Previous non-parallel disentanglement-based methods face severe training-testing inconsistency issues in XVC tasks due to language mismatch and the lack of multilingual parallel data, which inevitably compromise the quality of the synthesized speech. In this paper, we propose CASC-XVC, a zero-shot XVC method incorporating with content accordant (CA) and speaker contrastive (SC) losses. Specifically, this method adopts the framework of FreeVC-s as the backbone. We design a cross-lingual fine-tuning process employing pairs of utterances from speakers in different languages to update the modules used in the inference stage. A CA loss and an SC loss are introduced to deal with the lack of true parallel targets in the fine-tuning process. Moreover, we use shared self-supervised learning (SSL) representations across different languages along with information perturbation for content disentanglement. Both subjective and objective results on a bilingual (English and Chinese) dataset demonstrate that our approach achieves significant improvements in XVC tasks.

关键词： Perturbation methods Self-supervised learning Signal processing Acoustics Multilingual speech processing Faces

来源：评论

学校读者我要写书评

暂无评论

A Study of Multi-Scale Feature Learning From Pre-Trained Models on Speaker Verification

A Study of Multi-Scale Feature Learning From Pre-Trained Mod...

引用

International Conference on Acoustics, speech, and Signal processing (ICASSP)

作者： Shengyu Peng Wu Guo Jie Zhang Zuoliang Li Yu Guan Bin Gu Yang Ai National Engineering Research Center for Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China (USTC) Hefei China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

In this paper, a multi-scale feature fusion paradigm is proposed to fully exploit the power of the pre-trained models for text-independent speaker verification. It contains a front-end feature extractor and an enhanced ECAPA-TDNN backend in a cascade manner. The feature extractor incorporates local representations of the CNN layers as well as the global clues of the Transformer layers of the pre-trained models, which are combined to construct the multi-scale discriminative features. The outputs of the feature extractor are then fed into the back-end model (tailored from ECAPA-TDNN) to obtain the final speaker embedding. Results on VoxCeleb datasets validate the superiority of the proposed method with equal error rates of 0.633% and 0.457% on the official trials of Vox1-O using the base and large pre-trained models, respectively.

关键词： Representation learning Error analysis Signal processing Feature extraction Transformers Acoustics speech processing

来源：评论

学校读者我要写书评

暂无评论

KhmerFormer: Multi-Scale CNNs-Transformer with External Attention for Ancient Khmer Palm Leaf Isolated Glyph Classification

KhmerFormer: Multi-Scale CNNs-Transformer with External Atte...

引用

Asia-Pacific Signal and Information processing Association Annual Summit and Conference (APSIPA)

作者： Nimol Thuon Jun Du National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China Hefei China

ISBN: (数字)9798350367331

ISBN: (纸本)9798350367348

Ancient Khmer palm leaf manuscripts are invaluable cultural artifacts in Southeast Asia, especially in Cambodia. The preservation and study of these manuscripts are hindered by their complex glyph structures and the scarcity of resources for ancient languages. This paper introduces KhmerFormer, a hybrid model that combines Multi-Scale Convolutional Neural Networks (MS-CNNs) and Vision Transformers (ViTs) with External Attention (EA) mechanisms to enhance glyph classification tasks. Our approach includes a preprocessing algorithm for Khmer glyph enhancements (IEPalmV2) and leverages multi-scale feature extraction with EfficientNet, followed by integration into ViTs where traditional self-attention is replaced by external attention. This model is tailored to address the unique and intricate challenges presented by ancient Khmer manuscripts. Our study is evaluated in ICFHR 2018 and newly extracted datasets show significant performance enhancements, particularly in handling low-resource and imbalanced datasets. Our results highlight the potential of hybrid architectures in advancing the analysis of historical documents and providing robust solutions for their preservation.

关键词： Training Computer vision Text analysis Computational modeling Asia Computer architecture Linguistics Transformers Feature extraction Robustness

来源：评论

学校读者我要写书评

暂无评论

A Composite Predictive-Generative Approach to Monaural Universal speech Enhancement

IEEE Transactions on Audio, Speech and Language Processing

引用

IEEE Transactions on Audio, speech and language processing 2025年 33卷 2312-2325页

作者： Jie Zhang Haoyin Yan Xiaofei Li National Engineering Research Center for Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China (USTC) Hefei China School of Engineering Westlake University Hangzhou China

It is promising to design a single model that can suppress various distortions and improve speech quality, i.e., universal speech enhancement (USE). Compared to supervised learning-based predictive methods, diffusion-based generative models have shown greater potential due to the generative capacities from degraded speech with severely damaged information. However, artifacts may be introduced in highly adverse conditions, and diffusion models often suffer from a heavy computational burden due to many steps for inference. In order to jointly leverage the superiority of prediction and generation and overcome the respective defects, in this work we propose a universal speech enhancement model called PGUSE by combining predictive and generative modeling. Our model consists of two branches: the predictive branch directly predicts clean samples from degraded signals, while the generative branch optimizes the denoising objective of diffusion models. We utilize the output fusion and truncated diffusion scheme to effectively integrate predictive and generative modeling, where the former directly combines results from both branches and the latter modifies the reverse diffusion process with initial estimates from the predictive branch. Extensive experiments on several datasets verify the superiority of the proposed model over state-of-the-art baselines, demonstrating the complementarity and benefits of combining predictive and generative modeling.

关键词： Predictive models Diffusion models Computational modeling speech enhancement Training Diffusion processes Stochastic processes Standards Noise reduction Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Denoising-and-Dereverberation Hierarchical Neural Vocoder for Statistical Parametric speech Synthesis

Denoising-and-Dereverberation Hierarchical Neural Vocoder fo...

引用

作者： Ai, Yang Ling, Zhen-Hua Wu, Wei-Lu Li, Ang University of Science and Technology of China National Engineering Research Center of Speech and Language Information Processing Hefei230027 China National University of Defense Technology Hefei230037 China

This paper presents a denoising and dereverberation hierarchical neural vocoder (DNR-HiNet) to convert noisy and reverberant acoustic features into clean speech waveforms. The DNR-HiNet vocoder is built by modifying the amplitude spectrum predictor (ASP) in the original HiNet vocoder. This modified denoising and dereverberation ASP (DNR-ASP) can predict clean log amplitude spectra from input degraded acoustic features. To achieve this, the DNR-ASP first predicts the log amplitude spectra of noisy and reverberant speech, the log amplitude spectra of additive noise and the room impulse response (RIR) and then performs initial denoising and dereverberation by signal processing algorithms. The initially processed log amplitude spectra are then enhanced by another neural network to obtain the final clean log amplitude spectra. We also introduce a bandwidth extension model and a frequency resolution extension model into the DNR-ASP to further improve its performance. Finally, a statistical parametric speech synthesis (SPSS) method with DNR-HiNet is proposed to deal with the situation that the quality of target speaker's recordings is degraded by noise and reverberation. Experimental results indicate that the DNR-HiNet vocoder was able to generate denoised and dereverberated waveforms given noisy and reverberant acoustic features and outperformed the original HiNet vocoder and a few other neural vocoders. On speech enhancement tasks, its performance was competitive with several advanced speech enhancement methods. Furthermore, the SPSS method with DNR-HiNet achieved better quality of synthetic speech than the conventional approach which directly applied speech enhancement to the degraded adaptation data. © 2014 IEEE.

关键词： Hidden Markov models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：