检索结果-内蒙古大学图书馆

FLQ: Design and implementation of hybrid multi-base full logarithmic quantization neural network acceleration architecture based on FPGA

引用

SIGNAL PROCESSING-IMAGE COMMUNICATION 2025年 134卷

作者： Zhang, Longlong Hu, Xiang Liao, Xuan Zhou, Tong Peng, Yuanxi Natl Univ Def Technol Coll Comp Sci & Technol Changsha 410073 Peoples R China Natl Univ Def Technol Beijing Inst Adv Study Beijing 100000 Peoples R China

As deep neural network (DNN) models become more accurate, problems such as large model parameters and high computational complexity have become increasingly prominent, leading to a bottleneck in deploying them on resource-limited embedded platforms. In recent years, logarithm-based quantization techniques have shown great potential in reducing the inference cost of neural networks. However, current single-model log-quantization has reached an upper limit of classification performance, and little work has investigated hardware implementation of neural network quantization. In this paper, we propose a full logarithmic quantization (FLQ) mechanism that quantizes both weights and activation values into the logarithmic domain, compressing the parameters on AlexNet and VGG16 model by >6.4 times while maintaining an accuracy loss of within 2.5 % compared with benchmarking. Furthermore, we propose two optimization solutions for FLQ: activation segmented full logarithmic quantization (ASFLQ) and multi-ratio activation segmented full logarithmic quantization (Multi-ASFLQ), which can better balance the numerical representation range and quantization step. Under the condition of weight quantization of 5 bits and activation value quantization of 4 bits, the optimization methods proposed in this paper can improve the TOP1 of the VGG16 network model by 1 % and 1.6 %, respectively. Subsequently, we propose an implementation scheme of computing unit corresponding to the optimized FLQ mechanism above, which can not only convert multiplication operations into a shift operation but also integrate functions such as different ratio logarithmic bases and sparsity processing for activation, minimizing resource consumption as well as avoiding unnecessary calculations. Finally, we experiment with VGG19, Retnet50, and Densenet169 models, proving that the proposed method can achieve good performance under lower bit quantization. (c) 2001 Elsevier Science. All rights reserved

关键词： Deep neural network logarithmic quantization Computing units FPGAs

来源：评论

学校读者我要写书评

暂无评论

logarithmic quantization INDEX MODULATION: A PERCEPTUALLY BETTER WAY TO EMBED DATA WITHIN A COVER SIGNAL

LOGARITHMIC QUANTIZATION INDEX MODULATION: A PERCEPTUALLY BE...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Kalantari, Nima Khademi Ahadi, Seyed Mohammad Amir Kabir Univ Technol Dept Elect Engn Tehran 15914 Iran

ISBN: (纸本)9781424423538

In this paper, a new method for logarithmic quantization Index Modulation (QIM) is proposed. In this regard a logarithmic function is first applied to the host signal. Then the transformed signal is quantized using uniform quantization as conventional QIM to embed watermark data within. Finally using inverse transform the watermarked signal is obtained. The watermark extraction is performed using minimum distance decoder. The optimum parameter for data embedding with minimum quantization distortion is derived. Also the probability of error is analytically calculated and verified by simulation. Furthermore data hiding using secret key is proposed and the probability of error is obtained. Simulation results show that the proposed method outperforms the conventional QIM in terms of robustness when the perceptual quality of watermarked image for both methods are similar. Moreover, simulation shows that the proposed scheme has outstanding robustness in comparison with a recent quantization based data hiding method.

关键词： QIM logarithmic quantization digital watermarking Generalized Gaussian Distribution

来源：评论

学校读者我要写书评

暂无评论

MLogNet: A logarithmic quantization-Based Accelerator for Depthwise Separable Convolution

引用

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 2022年第12期41卷 5220-5231页

作者： Choi, Jooyeon Sim, Hyeonuk Oh, Sangyun Lee, Sugil Lee, Jongeun Ulsan Natl Univ Sci & Technol Dept Elect Engn Ulsan 689798 South Korea DeepinSight Seongnam 13449 South Korea Samsung Adv Inst Technol Suwon 16678 South Korea Ulsan Natl Univ Sci & Technol Dept Elect Engn Ulsan 44919 South Korea

In this article, we propose a novel logarithmic quantization-based deep neural network (DNN) architecture for depthwise separable convolution (DSC) networks. Our architecture is based on selective two-word logarithmic quantization (STLQ), which improves accuracy greatly over logarithmic-scale quantization while retaining the speed and area advantage of logarithmic quantization. On the other hand, it also comes with the synchronization problem due to variable-latency processing elements (PEs), which we address through a novel architecture and a compile-time optimization technique. Our architecture is dynamically reconfigurable to support various combinations of depthwise versus pointwise convolution layers efficiently. Our experimental results using layers from MobileNetV2 and ShuffleNetV2 demonstrate that our architecture is significantly faster and more area efficient than previous DSC accelerator architectures as well as previous accelerators utilizing logarithmic quantization.

关键词： Channel reordering deep learning processor depthwise separable convolution (DSC) logarithmic quantization variable-latency multiplication

来源：评论

学校读者我要写书评

暂无评论

State estimation of chaotic Lurie system with logarithmic quantization

引用

CHAOS SOLITONS & FRACTALS 2018年 112卷 141-148页

作者： Zheng, Juanhui Cui, Baotong Jiangnan Univ Key Lab Adv Proc Control Light Ind Minist Educ Wuxi 214122 Peoples R China Jiangnan Univ Sch IoT Engn Wuxi 214122 Peoples R China

In this paper, we address the problem of state estimation of the Lurie system via the communication channel in the case of only this system outputs available. A coder-decoder scheme combines with a logarithmic quantization to form a novel and reliable communication channel. The errors between Lurie system outputs and observer outputs are regarded as the feedback signals, which are transmitted into the observer though the communication channel. A sufficient condition for input-to-state stability is given for the boundedness of the error of state estimation. The results of two examples show the effectiveness and superiority of the proposed communication channel of the logarithmic quantization. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Lurie system State estimation logarithmic quantization Input-to-state stability

来源：评论

学校读者我要写书评

暂无评论

HLQ: Hardware-Friendly logarithmic quantization Aware Training for Power-Efficient Low-Precision CNN Models

引用

IEEE ACCESS 2024年 12卷 159611-159621页

作者： Choi, Dahun Park, Juntae Kim, Hyun Seoul Natl Univ Sci & Technol Res Ctr Elect & Informat Technol Dept Elect & Informat Engn Seoul 01811 South Korea

With the development of deep learning and graphics processing units (GPUs), various convolutional neural network (CNN)-based computer vision studies have been conducted. Because numerous computations are involved in the inference and training process of CNNs, research on network compression, including quantization, is being actively conducted along with the use of CNNs. Unlike the existing linear quantization, logarithmic quantization has the advantage that the multiply-accumulate (MAC) operation in the convolution (CONV) operation, which occupies most of the CNNs, can be replaced with the addition operation and is suitable for low-precision quantization. In this paper, we propose a logarithmic quantization aware training technique that effectively reduces quantization loss while maximizing the effect of reducing hardware resources and power consumption in the forward and backward propagation processes of the CNN. The proposed method minimizes the accuracy drop by allocating the rounding point with the least quantization loss for each specific training in the forward pass and propagates the optimized gradient by scaling the gradient of parameters with a high quantization loss in the backward pass. As a result of scratch training on the Tiny-ImageNet dataset using ResNet-18, 34, and 50, where both weights and activations are quantized to 4-bits through the proposed method, an improvement in accuracy of 0.88%, 0.48%, and 1.72%, respectively, can be achieved compared to that of the baseline (i.e., full-precision). In addition, as a result of synthesizing the CONV acceleration unit of ResNet-18 through RTL implementation, the proposed 4-bit quantization can achieve a power saving of 82.3% compared to the baseline (i.e., full-precision) when computing ResNet-18.

关键词： quantization (signal) Accuracy Training Propagation losses Convolutional neural networks Computational modeling Power demand Integrated circuit modeling Hardware Indexes Low power electronics Compression algorithms logarithmic quantization convolutional neural network low-power network compression

来源：评论

学校读者我要写书评

暂无评论

Base-Reconfigurable Segmented logarithmic quantization and Hardware Design for Deep Neural Networks

引用

JOURNAL OF SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY 2020年第11期92卷 1263-1276页

作者： Xu, Jiawei Huan, Yuxiang Jin, Yi Chu, Haoming Zheng, Li-Rong Zou, Zhuo Fudan Univ State Key Lab ASIC & Syst Shanghai Peoples R China

The growth in the size of deep neural network (DNN) models poses both computational and memory challenges to the efficient and effective implementation of DNNs on platforms with limited hardware resources. Our work on segmented logarithmic (SegLog) quantization, adopting both base-2 and base-root 2 logarithmic encoding, is able to reduce inference cost with a little accuracy penalty. However, weight distribution varies among layers in different DNN models, and requires different base-2 : base-root 2 ratios to reach the best accuracy. This means different hardware designs for the decoding and computing parts are required. This paper extends the idea of SegLog quantization by using layer-wise base-2 : base-root 2 ratio on weight quantization. The proposed base-reconfigurable segmented logarithmic (BRSLog) quantization is able to achieve 6.4x weight compression with 1.66% Top-5 accuracy drop on AlexNet at 5-bit resolution. An arithmetic element supporting BRSLog-quantified DNN inference is proposed to adapt to different base-2 : base- root 2 ratios. With root 2 approximation, the resource-consuming multipliers can be replaced by shifters and adders with only 0.54% accuracy penalty. The proposed arithmetic element is simulated in UMC 55nm Low Power Process, and it is 50.42% smaller in area and 55.60% lower in power consumption than the widely-used 16-bit fixed-point multiplier. Compared with equivalent SegLog arithmetic element designed for fixed base-2 : base-root 2 ratio, the base-reconfigurable part only increases the area by 22.96 mu m(2) and energy cost by 2.6 mu W.

关键词： logarithmic quantization Neural network Arithmetic element Embedded intelligence

来源：评论

学校读者我要写书评

暂无评论

A logarithmic quantization Index Modulation for Perceptually Better Data Hiding

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2010年第6期19卷 1504-1517页

作者： Kalantari, Nima Khademi Ahadi, Seyed Mohammad Amirkabir Univ Technol Dept Elect Engn Tehran Iran

In this paper, a novel arrangement for quantizer levels in the quantization Index Modulation (QIM) method is proposed. Due to perceptual advantages of logarithmic quantization, and in order to solve the problems of a previous logarithmic quantization-based method, we used the compression function of mu-Law standard for quantization. In this regard, the host signal is first transformed into the logarithmic domain using the mu-Law compression function. Then, the transformed data is quantized uniformly and the result is transformed back to the original domain using the inverse function. The scalar method is then extended to vector quantization. For this, the magnitude of each host vector is quantized on the surface of hyperspheres which follow logarithmic radii. Optimum parameter mu for both scalar and vector cases is calculated according to the host signal distribution. Moreover, inclusion of a secret key in the proposed method, similar to the dither modulation in QIM, is introduced. Performance of the proposed method in both cases is analyzed and the analytical derivations are verified through extensive simulations on artificial signals. The method is also simulated on real images and its performance is compared with previous scalar and vector quantization-based methods. Results show that this method features stronger a watermark in comparison with conventional QIM and, as a result, has better performance while it does not suffer from the drawbacks of a previously proposed logarithmic quantization algorithm.

关键词： Digital watermarking generalized Gaussian distribution logarithmic quantization quantization Index Modulation (QIM)

来源：评论

学校读者我要写书评

暂无评论

Spherical logarithmic quantization

引用

IEEE TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2010年第1期18卷 126-140页

作者： Matschkal, Bernd Huber, Johannes B. Inst Informat Transmiss D-91058 Erlangen Germany

Spherical logarithmic quantization (SLQ) is a vector quantization method for efficiently digitizing analog signals at a high dynamic range and with very low distortion while preserving the original waveform as closely as possible. SLQ is able to operate at a low data rate of, e.g., 2 bits per sample and at a very low signal delay of about ten samples, this corresponds to approximately 200 mu s for high-quality audio signals. The technique of SLQ is universally applicable (i.e., not restricted to, e.g., audio signals) and achieves an efficient digital representation of waveforms with high longterm as well as high segmental signal-to-noise ratios. The aim of this paper is to give a detailed description of the SLQ algorithm and to present simulation results on the performance of this new quantization scheme that combines several advantages. After a review of some important basic principles concerning quantization, linear prediction and multidimensional spheres, the SLQ encoder is described. To short vectors of signal samples which are represented in sphere coordinates, logarithmic quantization is applied to the radius and uniform quantization is applied to the angles. This results in the advantage of a constant signal-to-noise ratio over a very high dynamic range at a small loss with respect to the rate-distortion theory. In order to increase the signal-to-noise ratio by exploitation of correlations within the source signal, a solution for the problem of combining this vector quantization scheme with scalar adaptive differential pulse code modulation (ADPCM), i.e., ADPCM with sample by sample backward recursion is presented. Furthermore, an indexing scheme for the quantization cells covering the surface of a multidimensional unit sphere is presented and simulation results using different source signals are given.

关键词： Dynamic range logarithmic quantization low delay audio coding vector quantization

来源：评论

学校读者我要写书评

暂无评论

Hardware-Friendly logarithmic quantization with Mixed-Precision for MobileNetV2 4

Hardware-Friendly Logarithmic Quantization with Mixed-Precis...

引用

IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) - Intelligent Technology in the Post-Pandemic Era

作者： Choi, Dahun Kim, Hyun Seoul Natl Univ Sci & Technol Dept Elect & Informat Engn Res Ctr Elect & Informat Technol Seoul South Korea

ISBN: (数字)9781665409964

ISBN: (纸本)9781665409964

In a variety of computer vision applications, convolutional neural networks (CNNs) have achieved excellent accuracy. However, in order for a CNN to operate on embedded platforms such as mobile devices, hardware resources and power consumption must be reduced. Accordingly, research involving the application of low-precision quantization to lightweight networks, such as MobileNet, has attracted considerable attention. In particular, compared to linear quantization, logarithmic quantization can significantly reduce hardware resources by processing multiplication operations as addition operations when implementing a hardware accelerator. In this study, we propose a novel logarithmic weight quantization considering the characteristics of MobileNetV2, which is known to be notoriously difficult to quantize, and a mixed-precision quantization that minimizes accuracy loss by training the distribution range using the trainable parameter alpha. Experimental results show that the proposed method achieves accuracies greater than 1.47% and 2% on the CIFAR-10 and Tiny-ImageNet datasets, respectively, compared to the general log-scale quantization methods. As a result, the proposed method achieves a significant hardware resource reduction with only a slight degradation in performance when compared to the full precision (i.e., FP32), and achieves an additional power reduction effect of about 48% compared to linear scale quantization at the same precision.

关键词： Deep learning Convolutional Neural Network logarithmic quantization MobileNetV2

来源：评论

学校读者我要写书评

暂无评论

A Deep Look into logarithmic quantization of Model Parameters in Neural Networks 18

A Deep Look into Logarithmic Quantization of Model Parameter...

引用

10th International Conference on Advances in Information Technology (IAIT)

作者： Cai, Jingyong Takemoto, Masashi Nakajo, Hironori Tokyo Univ Agr & Technol Grad Sch Engn Tokyo Japan BeatCraft Inc Tokyo Japan Tokyo Univ Agr & Technol Inst Engn Tokyo Japan

ISBN: (纸本)9781450365680

Based on the fact that parameters of pre-trained neural networks naturally have non-uniform distributions, logarithmic quantization of network parameters achieves better classification results than linear quantization of the same resolution. In our practice, we found that the logarithmic quantization suffers huge accuracy decrease on small size neural networks. This is because the parameters of trained small neural networks are not highly concentrated around 0. In this paper, we analyse in depth the attributes of logarithmic quantization. In addition, existing compression algorithms highly rely on retraining which requires heavy computational power. In such a situation, we propose a new logarithmic quantization algorithm to mitigate the deterioration on neural networks which contain layers of small size. As the result, our method achieves the minimum accuracy loss on GoogLeNet after direct quantization compared to quantized counterparts.

关键词： Neural Networks Compression logarithmic quantization Computer Vision Deep Learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：