检索结果-内蒙古大学图书馆

Towards Lightweight Speaker Verification via Adaptive neural network quantization

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2024年 32卷 3771-3784页

作者： Liu, Bei Wang, Haoyu Qian, Yanmin Shanghai Jiao Tong Univ AI Inst Dept Comp Sci & Engn Auditory Cognit & Computat Acoust Lab Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ AI Inst MoE Key Lab Artificial Intelligence Shanghai 200240 Peoples R China

Modern speaker verification (SV) systems typically demand expensive storage and computing resources, thereby hindering their deployment on mobile devices. In this paper, we explore adaptive neural network quantization for lightweight speaker verification. Firstly, we propose a novel adaptive uniform precision quantization method which enables the dynamic generation of quantization centroids customized for each network layer based on k-means clustering. By applying it to the pre-trained SV systems, we obtain a series of quantized variants with different bit widths. To enhance low-bit quantized models, a mixed precision quantization algorithm along with a multi-stage fine-tuning (MSFT) strategy is further introduced. This approach assigns varying bit widths to different network layers. When bit combinations are determined, MSFT progressively quantizes and fine-tunes the network in a specific order. Finally, we design two distinct binary quantization schemes to mitigate performance degradation of 1-bit quantized models: the static and adaptive quantizers. Experiments on VoxCeleb demonstrate that lossless 4-bit uniform precision quantization is achieved on both ResNets and DF-ResNets, yielding a promising compression ratio of similar to 8. Moreover, compared to uniform precision approach, mixed precision quantization not only obtains additional performance improvements with a similar model size but also offers the flexibility to generate bit combination for any desirable model size. In addition, our suggested 1-bit quantization schemes remarkably boost the performance of binarized models. Finally, a thorough comparison with existing lightweight SV systems reveals that our proposed models outperform all previous methods by a large margin across various model size ranges.

关键词： Lightweight systems neural network quantization ResNet DF-ResNet speaker verification

来源：评论

学校读者我要写书评

暂无评论

EfficientQ: An efficient and accurate post-training neural network quantization method for medical image segmentation

引用

MEDICAL IMAGE ANALYSIS 2024年 97卷 103277页

作者： Zhang, Rongzhao Chung, Albert C. S. Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Hong Kong Peoples R China

Model quantization is a promising technique that can simultaneously compress and accelerate a deep neural network by limiting its computation bit-width, which plays a crucial role in the fast-growing AI industry. Despite model quantization's success in producing well-performing low-bit models, the quantization process itself can still be expensive, which may involve a long fine-tuning stage on a large, well-annotated training set. To make the quantization process more efficient in terms of both time and data requirements, this paper proposes a fast and accurate post-training quantization method, namely EfficientQ. We develop this new method with a layer-wise optimization strategy and leverage the powerful alternating direction method of multipliers (ADMM) algorithm to ensure fast convergence. Furthermore, a weight regularization scheme is incorporated to provide more guidance for the optimization of the discrete weights, and a self-adaptive attention mechanism is proposed to combat the class imbalance problem. Extensive comparison and ablation experiments are conducted on two publicly available medical image segmentation datasets, i.e., LiTS and BraTS2020, and the results demonstrate the superiority of the proposed method over various existing post training quantization methods in terms of both accuracy and optimization speed. Remarkably, with EfficientQ, the quantization of a practical 3D UNet only requires less than 5 min on a single GPU and one data sample. The source code is available at https://***/rongzhao-zhang/EfficientQ.

关键词： neural network quantization Post-training quantization Image segmentation Deep learning Model acceleration Model compression

来源：评论

学校读者我要写书评

暂无评论

FedQNN: A Computation-Communication-Efficient Federated Learning Framework for IoT With Low-Bitwidth neural network quantization

引用

IEEE INTERNET OF THINGS JOURNAL 2023年第3期10卷 2494-2507页

作者： Ji, Yu Chen, Lan Chinese Acad Sci Inst Microelect Beijing 100029 Peoples R China Beijing Municipal Commiss Sci & Technol Beijing Key Lab Three Dimens & Nanometer Integrate Beijing 100029 Peoples R China

Federated learning (FL) allows participants to train deep learning models collaboratively without disclosing their data to the server or any other participants, providing excellent value in the field of privacy-sensitive IoT. However, this distributed training paradigm requires clients to perform intensive computation for many iterations, which may exceed the capability of a typical IoT terminal with limited processing power, storage capacity, and energy budget. Heavy communication between the server and clients may also result in intolerant bandwidth requirements and energy consumption for many IoT systems. In this article, we introduce the FedQNN, a computation-communication-efficient FL framework for IoT scenarios. It is the first work that integrates ultralow-bitwidth quantization into the FL environment, allowing clients to perform lightweight fix-point computation efficiently with less power. Furthermore, both upstream and downstream data are significantly compressed for more efficient communication using a combination of sparsification and quantization strategies. We performed extensive experiments on a variety of data sets and models while comparing with other frameworks, and the results demonstrate that the proposed method can save up to 90% of our clients' computational energy, reduce model sizes by 30+ times, and significantly compress both communication bandwidth and transmitted data size while maintaining reasonable accuracy. The robustness against the non-independent and identically distributed (I.I.D.) condition is also validated.

关键词： quantization (signal) Computational modeling Internet of Things Training Computational efficiency Servers Federated learning Deep learning edge computing federated learning (FL) neural network quantization

来源：评论

学校读者我要写书评

暂无评论

LOW BIT neural network quantization FOR SPEAKER VERIFICATION

LOW BIT NEURAL NETWORK QUANTIZATION FOR SPEAKER VERIFICATION

引用

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wang, Haoyu Liu, Bei Wu, Yifei Chen, Zhengyang Qian, Yanmin Shanghai Jiao Tong Univ AI Inst Dept Comp Sci & Engn MoE Key Lab Artificial IntelligenceX LANCE Lab Shanghai Peoples R China

ISBN: (纸本)9798350302615

With the continuous development of deep neural networks (DNN) in recent years, the performance of speaker verification systems has been significantly improved with the application of Deeper ResNet architectures. However, these deeper models occupy more storage space in application. In this paper, we adopt Alternate Direction Methods of Multipliers (ADMM) to realize low-bit quantization on the original ResNets. Our goal is to explore the maximal quantization compression without evident degradation in model performance. We implement different uniform quantization for each convolution layer to achieve mixed precision quantization of the entire model. Moreover, the impact of batch normalization layers in ADMM training and layer sensibility to quantization are explored. In our experiments, the 8 bit quantized ResNet152 achieved comparable results to the full-precision one on Voxceleb 1, with only 45% of original model size. Besides, we find that shallow convolution layers are more sensitive to quantization. In addition, experimental results indicate that the model performance will be severely degraded if batch normalization layers are integrated into the convolution layer before the quantization training starts.

关键词： speaker verification neural network quantization model compression mixed precision quantization

来源：评论

学校读者我要写书评

暂无评论

Adaptive neural network quantization for Lightweight Speaker Verification 24

Adaptive Neural Network Quantization for Lightweight Speaker...

引用

Interspeech Conference

作者： Wang, Haoyu Liu, Bei Wu, Yifei Qian, Yanmin Shanghai Jiao Tong Univ MoE Key Lab Artificial Intelligence AI Inst X LANCE LabDept Comp Sci & Engn Shanghai Peoples R China

Recently, speaker verification systems benefit from deep neural networks and the size of speaker embedding encoder increases with these sophisticated architectures. Nevertheless, mobile devices have inadequate memory for oversized embedding extractors, thus demanding compact networks. In this paper, we explore neural network quantization for model compression. Specifically, we first propose a novel uniform quantization method based on K-Means clustering. Then, to further improve the small model performance, mixed precision quantization is introduced. Besides, we implement a multi-stage fine-tuning (MSFT) recipe to boost the accuracy of mixed-precision model. In experiments, the performance degradation of 4 bit quantized ResNet34 is negligible. Our quantized models outperform former model compression methods in terms of size and accuracy. In addition, mixed-precision quantization with MSFT strategy further improves the model performance.

关键词： speaker verification model compression neural network quantization mixed precision quantization

来源：评论

学校读者我要写书评

暂无评论

CEG4N: Counter-Example Guided neural network quantization Refinement 5th

CEG4N: Counter-Example Guided Neural Network Quantization Re...

引用

5th International Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS)

作者： Matos Jr, Joao Batista P. Bessa, Iury Manino, Edoardo Song, Xidan Cordeiro, Lucas C. Univ Fed Amazonas Manaus Amazonas Brazil Univeristy Manchester Machester England

ISBN: (纸本)9783031212215;9783031212222

neural networks are essential components of learning-based software systems. However, deploying neural networks in low-resource domains is challenging because of their high computing, memory, and power requirements. For this reason, neural networks are often quantized before deployment, but existing quantization techniques tend to degrade network accuracy. We propose Counter-Example Guided neural network quantization Refinement (CEG4N). This technique combines search-based quantization and equivalence verification: the former minimizes the computational requirements, while the latter guarantees that the network's output does not change after quantization. We evaluate CEG4N on a diverse set of benchmarks, including large and small networks. Our technique successfully quantizes the networks in our evaluation while producing models with up to 72% better accuracy than state-of-the-art techniques.

关键词： Robust quantization neural network quantization neural network equivalence Counter example guided optimization

来源：评论

学校读者我要写书评

暂无评论

A 4-bit Integer-Only neural network quantization Method Based on Shift Batch Normalization

A 4-bit Integer-Only Neural Network Quantization Method Base...

引用

IEEE International Symposium on Circuits and Systems (ISCAS)

作者： Guo, Qingyu Cui, Xiaoxin Zhang, Jian Zhang, Aifei Guo, Xinjie Wang, Yuan Peking Univ Sch Integrated Circuits Key Lab Microelect Devices & Circuits MoE Beijing 100871 Peoples R China Beijing Zhicun WITIN Technol Corp Ltd Beijing 100191 Peoples R China Peking Univ Beijing Lab Future IC Technol & Sci Beijing 100871 Peoples R China

ISBN: (纸本)9781665484855

neural networks are powerful, but at the cost of huge amounts of computation. Deploying neural networks on edge devices is especially challenging. quantization is a possible solution to alleviate the huge cost, while most quantization methods are not sufficiently hardware-friendly. In this paper, we proposed an integer-only quantization method. With no division or big integer multiplication, this quantization method is suitable to be deployed on co-designed hardware platforms. We applied 4-bit quantization on some classical networks and corresponding datasets. On MNIST, CIFAR10 and CIFAR100, quantization networks perform as well as original networks. On SpeechCommands, accuracy error induced by quantization is 0.16%. We also deployed quantized networks under OpenCL framework and on a flash-based in-memory-computing chip to verify this method's feasibility.

关键词： integer-only neural network quantization

来源：评论

学校读者我要写书评

暂无评论

Auto-tuning neural network quantization Framework for Collaborative Inference Between the Cloud and Edge 27th

Auto-tuning Neural Network Quantization Framework for Collab...

引用

27th International Conference on Artificial neural networks (ICANN)

作者： Li, Guangli Liu, Lei Wang, Xueying Dong, Xiao Zhao, Peng Feng, Xiaobing Chinese Acad Sci Inst Comp Technol State Key Lab Comp Architecture Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China

ISBN: (纸本)9783030014186

Recently, deep neural networks (DNNs) have been widely applied in mobile intelligent applications. The inference for the DNNs is usually performed in the cloud. However, it leads to a large overhead of transmitting data via wireless network. In this paper, we demonstrate the advantages of the cloud-edge collaborative inference with quantization. By analyzing the characteristics of layers in DNNs, an auto-tuning neural network quantization framework for collaborative inference is proposed. We study the effectiveness of mixed-precision collaborative inference of state-of-the-art DNNs by using ImageNet dataset. The experimental results show that our framework can generate reasonable network partitions and reduce the storage on mobile devices with trivial loss of accuracy.

关键词： neural network quantization Auto-tuning framework Edge computing Collaborative inference

来源：评论

学校读者我要写书评

暂无评论

LIGHT-WEIGHT VISUALVOICE: neural network quantization ON AUDIO VISUAL SPEECH SEPARATION

LIGHT-WEIGHT VISUALVOICE: NEURAL NETWORK QUANTIZATION ON AUD...

引用

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wu, Yifei Li, Chenda Qian, Yanmin Shanghai Jiao Tong Univ Dept Comp Sci & Engn X LANCE Lab MoE Key Lab Artificial IntelligenceAI Inst Shanghai Peoples R China

ISBN: (纸本)9798350302615

As multi-modal systems show superior performance on more tasks, the huge amount of computational resources they need becomes one of the critical problems to be solved. In this work, we explore neural network quantization methods to compress the resource requirement of VisualVoice, a state-of-the-art audio-visual speech separation system. The model is firstly fine-tuned by an ADMM-based quantization-aware training approach to produce the fixed-precision quantized version. Then three strategies, including manual selection, Hessian trace-based selection and KL divergence-based greedy search are explored to find the optimal mixed-precision setting of the model. The result shows that by applying the optimal strategy, we obtain a satisfying trade-off between space, speed and performance for the final system. The KL divergence-based strategy reaches 7.2 dB in SDR at 3-bit equivalent setup, which outperforms the fixed-precision setup and the other two mixed-precision strategies. Moreover, we also discuss the influence caused by quantizing different parts of the multi-modal system.

关键词： Multi-modal audio-visual speech separation neural network quantization mixed-precision quantization

来源：评论

学校读者我要写书评

暂无评论

MedQ: Lossless ultra-low-bit neural network quantization for medical image segmentation

引用

MEDICAL IMAGE ANALYSIS 2021年 73卷 102200-102200页

作者： Zhang, Rongzhao Chung, Albert C. S. Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Lo Kwee Seong Med Image Anal Lab Hong Kong Peoples R China

Implementing deep convolutional neural networks (CNNs) with boolean arithmetic is ideal for eliminating the notoriously high computational expense of deep learning models. However, although lossless model compression via weight-only quantization has been achieved in previous works, it is still an open problem about how to reduce the computation precision of CNNs without losing performance, especially for medical image segmentation tasks where data dimension is high and annotation is scarce. This paper presents a novel CNN quantization framework that can squeeze a deep model (both parameters and activation) to extremely low bitwidth, e.g., 1 similar to 2 bits, while maintaining its high performance. In the new method, we first design a strong baseline quantizer with an optimizable quantization range. Then, to relieve the back-propagation difficulty caused by the discontinuous quantization function, we design a radical residual connection scheme that allows gradients to flow through every quantized layer freely. Moreover, a tanh-based derivative function is used to further boost gradient flow and a distributional loss is employed to regularize the model output. Extensive experiments and ablation studies are conducted on two well-established public 3D segmentation datasets, i.e., BRATS2020 and LiTS. Experimental results evidence that our framework not only outperforms state-of-the-art quantization approaches significantly, but also achieves lossless performance on both datasets with ternary (2-bit) quantization. (C) 2021 Elsevier B.V. All rights reserved.

关键词： neural network quantization Image segmentation Deep learning Model acceleration Model compression

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：