检索结果-内蒙古大学图书馆

TFF-codec: A High Fidelity End-to-End Neural audio codec

CIRCUITS SYSTEMS AND SIGNAL PROCESSING 2025年第6期44卷 4013-4032页

作者： Zhao, Yuhao Jia, Maoshen Ru, Jiawei Wang, Lizhong Wen, Liang Beijing Univ Technol Sch Informat Sci & Technol Beijing 100124 Peoples R China Samsung R&D Inst China Beijing Beijing 100028 Peoples R China

audio Coding has made significant progress with the development of deep neural networks. Recently, neural speech codecs based on vector quantized variational autoencoder have become increasingly popular among researchers due to their elegant design and superior performance, but their application to high bitrate audio coding has not been further expanded. In this paper, we propose a novel high fidelity end-to-end neural audio codec called time frequency fusion codec (TFF-codec), which is capable of high-quality reconstruction of 32 kHz audio in the time-frequency domain at 48 and 64 kbps. In this paper, a dual-path time-frequency filtering module is proposed to capture the local structure of the spectrogram and the long-term time dependence between consecutive frames. The architecture of the proposed codec is composed of encoder, the time-frequency filtering module, vector quantizer and decoder. First, the input audio is fed into the encoder to obtain its potential representation. Then, it is modeled in the frequency domain in the time-frequency filtering module. Subsequently, it is further compressed by a vector quantizer. Finally, the reconstructed audio is obtained by the decoder. We also use a combination of multiple loss functions in TFF-codec to ensure that the reconstructed audio is balanced in terms of objective metrics and subjective listening experience. To evaluate the performance of TFF-codec, comparative experiments are conducted with the traditional audio codec Opus and several recent neural audio codecs. Both subjective and objective evaluation tests demonstrate the superiority of our proposed method.

关键词： audio codec End to end neural network Auto encoder High fidelity audio generation

来源：评论

学校读者我要写书评

暂无评论

Semanticodec: An Ultra Low Bitrate Semantic audio codec for General Sound

引用

IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 2024年第8期18卷 1448-1461页

作者： Liu, Haohe Xu, Xuenan Yuan, Yi Wu, Mengyue Wang, Wenwu Plumbley, Mark D. Univ Surrey Ctr Vis Speech & Signal Proc CVSSP Guildford GU2 7XH England Shanghai Jiao Tong Univ Dept Comp Sci & Engn Shanghai 200240 Peoples R China

Large languagemodels (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modelling techniques to audio data. However, traditional codecs often operate at high bitrates or within narrow domains such as speech and lack the semantic clues required for efficient language modelling. Addressing these challenges, we introduce Semanticodec, a novel codec designed to compress audio into fewer than a hundred tokens per second across diverse audio types, including speech, general sound, and music, without compromising quality. Semanticodec features a dual-encoder architecture: a semantic encoder using a self-supervised pre-trained audio Masked Autoencoder (audioMAE), discretized using k-means clustering on extensive audio data, and an acoustic encoder to capture the remaining details. The semantic and acoustic encoder outputs are used to reconstruct audio via a diffusion-model-based decoder. Semanticodec is presented in three variants with token rates of 25, 50, and 100 per second, supporting a range of ultra-low bit rates between 0.31 kbps and 1.40 kbps. Experimental results demonstrate that Semanticodec significantly outperforms the state-of-the-art Descript codec on reconstruction quality. Our results also suggest that Semanticodec contains significantly richer semantic information than all evaluated state-of-the-art audio codecs, even at significantly lower bitrates.

关键词： codecs Semantics Bit rate Computational modeling Acoustics Diffusion models Speech coding Decoding Vector quantization Accuracy audio codec semantic low bitrate

来源：评论

学校读者我要写书评

暂无评论

NDVQ: ROBUST NEURAL audio codec WITH NORMAL DISTRIBUTION-BASED VECTOR QUANTIZATION

NDVQ: ROBUST NEURAL AUDIO CODEC WITH NORMAL DISTRIBUTION-BAS...

引用

2024 Spoken Language Technology Workshop

作者： Niu, Zhikang Chen, Sanyuan Zhou, Long Ma, Ziyang Chen, Xie Liu, Shujie Shanghai Jiao Tong Univ AI Inst MoE Key Lab Artificial Intelligence X LANCE Lab Shanghai Peoples R China Microsoft Res Asia Beijing Peoples R China

ISBN: (纸本)9798350392265;9798350392258

Built upon vector quantization (VQ), discrete audio codec models have achieved great success in audio compression and autoregressive audio generation. However, existing models face substantial challenges in perceptual quality and signal distortion, especially when operating in extremely low bandwidth, rooted in the sensitivity of the VQ codebook to noise. This degradation poses significant challenges for several downstream tasks, such as codec-based speech synthesis. To address this issue, we propose a novel VQ method, Normal Distribution-based Vector Quantization (NDVQ), by introducing an explicit margin between the VQ codes via learning a variance. Specifically, our approach involves mapping the waveform to a latent space and quantizing it by selecting the most likely normal distribution, with each codebook entry representing a unique normal distribution defined by its mean and variance. Using these distribution-based VQ codec codes, a decoder reconstructs the input waveform. NDVQ is trained with additional distribution-related losses, alongside reconstruction and discrimination losses. Experiments demonstrate that NDVQ outperforms existing audio compression baselines, such as Encodec, in terms of audio quality and zero-shot TTS, particularly in very low bandwidth scenarios.

关键词： audio Compression audio codec Normal Distribution Vector Quantization Text to Speech

来源：评论

学校读者我要写书评

暂无评论

LIGHTcodec: A HIGH FIDELITY NEURAL audio codec WITH LOW COMPUTATION COMPLEXITY 49

LIGHTCODEC: A HIGH FIDELITY NEURAL AUDIO CODEC WITH LOW COMP...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Xu, Liang Wang, Jing Zhang, Jianqian Xie, Xiang Beijing Inst Technol Sch Informat & Elect Beijing Peoples R China

ISBN: (纸本)9798350344868;9798350344851

The audio codec is one of the core modules in audio communication for real-time transmission. With the development of neural networks, end-to-end audio codecs have emerged and demonstrated effects beyond conventional codecs. However, current neural network-based codecs have the weakness of high computational complexity, and the performance of these methods decreases rapidly after decreasing the complexity, which is not conducive to deployment under low computational resources. In this paper, a low-complexity audio codec is proposed. To realize the low complexity of the model with high quality, a structure based on frequency band division is designed, which is implemented using a within band-across band interaction (WBABI) module to learn the features across and within the subband. Further, we propose a new quantization-compensation module, which reduces the quantization error by 90%. The experimental results show that for audio with a sample rate of 24kHz, the model shows excellent performance at 3 similar to 6kbps compared to other codecs, and the complexity is only 0.8 Giga Multiply-Add Operations per Second(GMACs).

关键词： audio codec Vector quantization Low complexity

来源：评论

学校读者我要写书评

暂无评论

Improving audio codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model 25

Improving Audio Codec-based Zero-Shot Text-to-Speech Synthes...

引用

25th Interspeech Conference

作者： Xue, Jinlong Deng, Yayue Han, Yicheng Gao, Yingming Li, Ya Beijing Univ Posts & Telecommun Beijing Peoples R China

Recent advances in large language models (LLMs) and development of audio codecs greatly propel the zero-shot TTS. They can synthesize personalized speech with only a 3-second speech of an unseen speaker as acoustic prompt. However, they only support short speech prompts and cannot leverage longer context information, as required in audiobook and conversational TTS scenarios. In this paper, we introduce a novel audio codec-based TTS model to adapt context features with multiple enhancements. Inspired by the success of Qformer, we propose a multi-modal context-enhanced Qformer (MMCE-Qformer) to utilize additional multi-modal context information. Besides, we adapt a pretrained LLM to leverage its understanding ability to predict semantic tokens, and use a SoundStorm to generate acoustic tokens thereby enhancing audio quality and speaker similarity. The extensive objective and subjective evaluations show that our proposed method outperforms baselines across various context TTS scenarios.

关键词： zero-shot speech synthesis audio codec context modeling

来源：评论

学校读者我要写书评

暂无评论

SCOREDEC: A PHASE-PRESERVING HIGH-FIDELITY audio codec WITH A GENERALIZED SCORE-BASED DIFFUSION POST-FILTER 49

SCOREDEC: A PHASE-PRESERVING HIGH-FIDELITY AUDIO CODEC WITH ...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wu, Yi-Chiao Markovic, Dejan Krenn, Steven Gebru, Israel D. Richard, Alexander Codec Avatars Lab Pittsburgh PA 19104 USA

ISBN: (纸本)9798350344868;9798350344851

Although recent mainstream waveform-domain end-to-end (E2E) neural audio codecs achieve impressive coded audio quality with a very low bitrate, the quality gap between the coded and natural audio is still significant. A generative adversarial network (GAN) training is usually required for these E2E neural codecs because of the difficulty of direct phase modeling. However, such adversarial learning hinders these codecs from preserving the original phase information. To achieve human-level naturalness with a reasonable bitrate, preserve the original phase, and get rid of the tricky and opaque GAN training, we develop a score-based diffusion post-filter (SPF) in the complex spectral domain and combine our previous audioDec with the SPF to propose ScoreDec, which can be trained using only spectral and score-matching losses. Both the objective and subjective experimental results show that ScoreDec with a 24 kbps bitrate encodes and decodes full-band 48 kHz speech with human-level naturalness and well-preserved phase information.

关键词： audio codec phase-preserving codec codec post-filter score-based generative model human-level naturalness

来源：评论

学校读者我要写书评

暂无评论

Towards audio codec-based Speech Separation 25

Towards Audio Codec-based Speech Separation

引用

25th Interspeech Conference

作者： Qi, Jia Zhao, Shengkui Ng, Dianwen Chng, Eng Siong Bin Mai Nanyang Technol Univ Interdisciplinary Grad Programme Alibaba NTU Singapore JRI Singapore Singapore Nanyang Technol Univ Singapore Singapore Alibaba Grp Hangzhou Peoples R China

Recent improvements in neural audio codec (NAC) models have generated interest in adopting pre-trained codecs for a variety of speech processing applications to take advantage of the efficiencies gained from high compression, but these have yet been applied to the speech separation (SS) task. SS can benefit from high compression because the compute required for traditional SS models makes them impractical for many edge computing use cases. However, SS is a waveform-masking task where compression tends to introduce distortions that severely impact performance. Here we propose a novel task of audio codec-based SS, where SS is performed within the embedding space of a NAC, and propose a new model, codecformer, to address this task. At inference, codecformer achieves a 52x reduction in MAC while producing separation performance comparable to a cloud deployment of Sepformer. This method charts a new direction for performing efficient SS in practical scenarios.

关键词： speech separation audio codec resource efficient neural audio compression

来源：评论

学校读者我要写书评

暂无评论

Implementation of Low-Bit Rate audio codec, codec2, inVerilog on Modern FPGAS

Implementation of Low-Bit Rate Audio Codec, Codec2, inVerilo...

引用

作者： Sampath Kumar, Santhiya Miami University

学位级别：硕士

audio compression codecs are an important application in the Internet of Things (IoT) where small sensing devices will gather voice signals, but then need to transmit the information to aggregating servers at low cost. In this work, we implement and evaluate a hardware implementation of the codec2, a lossy speech compression codec in Verilog and map it to an Intel CycloneIV FPGA. We describe the details of our implementation approach, including how we converted the C code of codec2, how we represent data inside the hardware implementation and the associated cost of this implementation on a real FPGA. We then analyze our implementation compared to a microprocessor implementation to observe what performance we get on an FPGA versus a microprocessor. Our hardware implementation of codec2 is qualitatively the same in terms of hearing the spoken transmission and has an error rate of 6.55 bits per frame (48 bits) and is 1.73 times faster than the microprocessor implementation.

关键词： Computer Engineering Electrical Engineering FPGA codec2 Verilog RTL DSP fixed-point audio codec Lossy Compression

来源：评论

学校读者我要写书评

暂无评论

Performance Evaluation of audio Coding by Amalgam AAC and FLAC audio codec using MDCT and INTMDCT Algorithm

引用

INTERNATIONAL JOURNAL OF ELECTRONICS AND TELECOMMUNICATIONS 2019年第3期65卷 533-539页

作者： Dhas, M. Davidson Kamala Priyadharsini, R. Mepco Schlenk Engn Coll ECE Sivakasi India

The MDCT and IntMDCT Algorithm is widely utilized is audio coding. By lifting scheme or rounding operation IntegerMDCT is evolved from Modified Discrete Cosine Transform. This method acquire the properties of MDCT and contribute excelling invertiblity and good spectral mean. In this paper we discuss about the audio codec like AAC and FLAC using MDCT and Integer MDCT algorithm and to find which algorithm shows better Compression Ratio(CR). The confines of this task is to hybriding lossy and lossless audio codec with diminished bit rate but with finer sound quality. Certainly the quality of the audio is figure out by Subjective and Objective testing which is in terms of MOS (Mean opinion square), ABx and some of the hearing aid testing methodology like PEAQ(Perceptual Evaluation audio Quality) and ODG(Objective Difference Grade) is followed. Execution measure, that is Compression Ratio(CR) and Sound Pressure Level (SPL) is approximated.

关键词： audio codec Advanced audio Coding (AAC) Free Lossless audio codec (FLAC) Modified Discrete Cosine Transform (MDCT) Integer Modified Discrete Cosine Transform (IntMDCT) Mean Opinion square (MOS) Perceptual Evaluation audio Quality (PEAQ)

来源：评论

学校读者我要写书评

暂无评论

audio codec and Digital Signal Processor for an Electrical Impedance Tomography System 1

引用

2nd Latin American Conference on Bioimpedance (CLABIO)

作者： Alfaro, N. Arregui, M. Martinucci, F. Santos, E. Simini, F. Univ Republica Fac Med & Ingn Nucleo Ingn Biomed Montevideo Uruguay

ISBN: (数字)9789812879288

ISBN: (纸本)9789812879288;9789812879264

Electrical Impedance Tomography (EIT) can estimate thorax fluid/air content and distribution. Its use in critically ill patients is promising and may prove clinically useful. Currents are injected and voltages measured in the thorax boundary. The measurements are used to reconstruct tomographic images. An EIT system was implemented based on evaluation board OMAP-L137 (Spectrum Digital) and Howland current source. The board main components are the OMAP-L137 processor with DSP C6747 and AIC 3106 audio codec (Texas Instrument). Fewer than 5% difference in voltages against the oscilloscope was measured in the range between 42mV and 1.5V. The entire system showed 8.53% difference when measuring the real part of impedance.

关键词： Electrical Impedance Tomography audio codec Digital Signal Processor Synchronous Demodulation Howland Current Source

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：