检索结果-内蒙古大学图书馆

Optimized Block-Based Lossy Image compression Technique for Wireless Sensor networks

IEEE ACCESS 2023年 11卷 131245-131259页

作者： Lungisani, Bose A. Zungeru, Adamu M. Lebekwe, Caspar K. Yahya, Abid Botswana Int Univ Sci & Technol Dept Elect Comp & Telecommun Engn Palapye Botswana

Traditionally, image compression algorithms have primarily focused on optimizing storage without considering resource-constrained applications, such as wireless sensor networks (WSNs). However, for practical application in WSNs, a balanced trade-off between compression ratio, distortion, and energy consumption is crucial to improve the network lifetime while maintaining acceptable image reconstruction quality at lower bit and error rates. Previous studies have focused on higher image compression rates to address storage limitations rather than on the optimization of WSNs. In addition, most previous research on image compression for WSNs either requires an error-bound mechanism or compromises the trade-off between compression ratios and reconstructed image quality measured using image quality assessment metrics, such as root mean square error (RMSE), and coefficient of determination (R2), leading to a reduced network lifetime and uncontrolled reconstructed image quality that is not application-specific. Therefore, we present an optimized block-based image compression algorithm for WSNs with a relative error-bound mechanism that adapts to a given dataset to improve reconstruction fidelity and energy consumption at higher compression ratios. A comparison of our proposed algorithm with existing algorithms demonstrated that using a convolutional variational autoencoder and relative error-bound mechanism leads to a significant trade-off in distortion, compression ratio, and energy consumption in WSNs. Our results demonstrated that an average reconstruction fidelity of more than 90% was achieved using image quality evaluation metrics at compression ratios of 60% or more. Furthermore, more than 50% energy conservation at compression ratios greater than 60% from image compression was achieved compared with the transmission of raw data within a WSN.

关键词： Image quality Training Rate distortion theory Image compression neural networks compression reconstruction fidelity wireless sensor networks

来源：评论

学校读者我要写书评

暂无评论

Quantized neural networks: Training neural networks with Low Precision Weights and Activations

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2018年第154期18卷 1-30页

作者： Hubara, Itay Courbariaux, Matthieu Soudry, Daniel El-Yaniv, Ran Bengio, Yoshua Technion Israel Inst Technol Dept Elect Engn Haifa Israel Univ Montreal Dept Comp Sci Montreal PQ Canada Univ Montreal Dept Stat Montreal PQ Canada Columbia Univ Dept Stat New York NY USA Technion Israel Inst Technol Dept Comp Sci Haifa Israel

We introduce a method to train Quantized neural networks (QNNs) neural networks with extremely low precision (e.g., 1-bit) weights and activations, at run-time. At traintime the quantized weights and activations are used for computing the parameter gradients. During the forward pass, QNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations. As a result, power consumption is expected to be drastically reduced. We trained QNNs over the MNIST, CIFAR-10, SVHN and ImageNet datasets. The resulting QNNs achieve prediction accuracy comparable to their 32-bit counterparts. For example, our quantized version of AlexNet with 1-bit weights and 2-bit activations achieves 51% top-1 accuracy. Moreover, we quantize the parameter gradients to 6-bits as well which enables gradients computation using only bit-wise operation. Quantized recurrent neural networks were tested over the Penn Treebank dataset, and achieved comparable accuracy as their 32-bit counterparts using only 4-bits. Last but not least, we programmed a binary matrix multiplication GPU kernel with which it is possible to run our MNIST QNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The QNN code is available online.

关键词： deep learning neural networks compression energy efficient neural networks computer vision language models

来源：评论

学校读者我要写书评

暂无评论

Interpretable Task-inspired Adaptive Filter Pruning for neural networks Under Multiple Constraints

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2024年第6期132卷 2060-2076页

作者： Guo, Yang Gao, Wei Li, Ge Peking Univ Sch Elect & Comp Engn Shenzhen Peoples R China

Existing methods for filter pruning mostly rely on specific data-driven paradigms but lack the interpretability. Besides, these approaches usually assign layer-wise compression ratios automatically only under given FLOPs by neural architecture search algorithms or just manually, which are short of efficiency. In this paper, we propose a novel interpretable task-inspired adaptive filter pruning method for neural networks to solve the above problems. First, we treat filters as semantic detectors and develop the task-inspired importance criteria by evaluating correlations between input tasks and feature maps, and observing the information flow through filters between adjacent layers. Second, we refer to the human neurobiological mechanism for the better interpretability, where the retained first layer filters act as individual information receivers. Third, inspired by the phenomenon that each filter has a deterministic impact on FLOPs and network parameters, we provide an efficient adaptive compression ratio allocation strategy based on differentiable pruning approximation under multiple budget constraints, as well as considering the performance objective. The proposed method is validated with extensive experiments on the state-of-the-art neural networks, which significantly outperforms all the existing filter pruning methods and achieves the best trade-off between neural network compression and task performance. With ResNet-50 on ImageNet, our approach reduces 75.49% parameters and 70.90% FLOPs, only suffering from 2.31% performance degradation.

关键词： Interpretable Task-inspired Adaptive Filter pruning neural networks compression Multiple constraints

来源：评论

学校读者我要写书评

暂无评论

Hybrid Approach for Efficient Quantization of Weights in Convolutional neural networks

Hybrid Approach for Efficient Quantization of Weights in Con...

引用

IEEE International Conference on Big Data and Smart Computing (BigComp)

作者： Seo, Sanghyun Kim, Juntae Dongguk Univ Dept Comp Engn Seoul South Korea

ISBN: (纸本)9781538636497

Convolutional neural networks(CNN) have achieved outstanding results in the fields of image recognition which classifies objects in the input images. In the deep neural networks such as CNN, the number of layers and the number of neurons in each layer are large. In other words, the deep neural networks requires relatively large storage space and calculation process. However, in embedded devices for object recognition in autonomous vehicles, large storage space and high computational complexity are constraints. For this reasons, various methodologies have been proposed to apply CNN to small embedded hardware such as mobile devices, FPGA and ASIC efficiently. In this paper, we quantize the weights of AlexNet without a large drop in accuracy by using a hybrid quantizer using uniform quantizer and k-means clustering.

关键词： neural networks compression Convolutional neural networks Weights Quantization Hybrid Quantizer

来源：评论

学校读者我要写书评

暂无评论

A Deep Look into Logarithmic Quantization of Model Parameters in neural networks 18

A Deep Look into Logarithmic Quantization of Model Parameter...

引用

10th International Conference on Advances in Information Technology (IAIT)

作者： Cai, Jingyong Takemoto, Masashi Nakajo, Hironori Tokyo Univ Agr & Technol Grad Sch Engn Tokyo Japan BeatCraft Inc Tokyo Japan Tokyo Univ Agr & Technol Inst Engn Tokyo Japan

ISBN: (纸本)9781450365680

Based on the fact that parameters of pre-trained neural networks naturally have non-uniform distributions, logarithmic quantization of network parameters achieves better classification results than linear quantization of the same resolution. In our practice, we found that the logarithmic quantization suffers huge accuracy decrease on small size neural networks. This is because the parameters of trained small neural networks are not highly concentrated around 0. In this paper, we analyse in depth the attributes of logarithmic quantization. In addition, existing compression algorithms highly rely on retraining which requires heavy computational power. In such a situation, we propose a new logarithmic quantization algorithm to mitigate the deterioration on neural networks which contain layers of small size. As the result, our method achieves the minimum accuracy loss on GoogLeNet after direct quantization compared to quantized counterparts.

关键词： neural networks compression Logarithmic Quantization Computer Vision Deep Learning

来源：评论

学校读者我要写书评

暂无评论

No Fine-Tuning, No Cry: Robust SVD for Compressing Deep networks

引用

SENSORS 2021年第16期21卷 5599-5599页

作者： Tukan, Murad Maalouf, Alaa Weksler, Matan Feldman, Dan Univ Haifa Dept Comp Sci Robot & Big Data Lab IL-3498838 Haifa Israel Samsung Res Israel IL-4659071 Herzliyya Israel

A common technique for compressing a neural network is to compute the k-rank l2 approximation Ak of the matrix A is an element of Rnxd via SVD that corresponds to a fully connected layer (or embedding layer). Here, d is the number of input neurons in the layer, n is the number in the next one, and Ak is stored in O((n+d)k) memory instead of O(nd). Then, a fine-tuning step is used to improve this initial compression. However, end users may not have the required computation resources, time, or budget to run this fine-tuning stage. Furthermore, the original training set may not be available. In this paper, we provide an algorithm for compressing neural networks using a similar initial compression time (to common techniques) but without the fine-tuning step. The main idea is replacing the k-rank l2 approximation with lp, for p is an element of[1,2], which is known to be less sensitive to outliers but much harder to compute. Our main technical result is a practical and provable approximation algorithm to compute it for any p >= 1, based on modern techniques in computational geometry. Extensive experimental results on the GLUE benchmark for compressing the networks BERT, DistilBERT, XLNet, and RoBERTa confirm this theoretical advantage.

关键词： matrix factorization neural networks compression robust low rank approximation Lowner ellipsoid

来源：评论

学校读者我要写书评

暂无评论

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

引用

SENSORS 2023年第3期23卷 1595-1595页

作者： Fanariotis, Anastasios Orphanoudakis, Theofanis Kotrotsios, Konstantinos Fotopoulos, Vassilis Keramidas, George Karkazis, Panagiotis Hellen Open Univ Sch Sci & Technol Digital Syst & Media Comp Lab Patras 26334 Greece Aristotle Univ Thessaloniki Dept Informat Thessaloniki 54124 Greece Univ West Att Dept Informat & Comp Engn Athens 12243 Greece

Computing has undergone a significant transformation over the past two decades, shifting from a machine-based approach to a human-centric, virtually invisible service known as ubiquitous or pervasive computing. This change has been achieved by incorporating small embedded devices into a larger computational system, connected through networking and referred to as edge devices. When these devices are also connected to the Internet, they are generally named Internet-of-Thing (IoT) devices. Developing Machine Learning (ML) algorithms on these types of devices allows them to provide Artificial Intelligence (AI) inference functions such as computer vision, pattern recognition, etc. However, this capability is severely limited by the device's resource scarcity. Embedded devices have limited computational and power resources available while they must maintain a high degree of autonomy. While there are several published studies that address the computational weakness of these small systems-mostly through optimization and compression of neural networks- they often neglect the power consumption and efficiency implications of these techniques. This study presents power efficiency experimental results from the application of well-known and proven optimization methods using a set of well-known ML models. The results are presented in a meaningful manner considering the "real world" functionality of devices and the provided results are compared with the basic "idle" power consumption of each of the selected systems. Two different systems with completely different architectures and capabilities were used providing us with results that led to interesting conclusions related to the power efficiency of each architecture.

关键词： machine learning power efficiency edge computing autonomous systems neural networks compression

来源：评论

学校读者我要写书评

暂无评论

Knowledge distillation via instance-level sequence learning

引用

KNOWLEDGE-BASED SYSTEMS 2021年 233卷 107519-107519页

作者： Zhao, Haoran Sun, Xin Dong, Junyu Dong, Zihe Li, Qiong Ocean Univ China Coll Informat Sci & Engn Qingdao Peoples R China Tech Univ Munich Dept Aerosp & Geodesy Munich Germany

Recently, distillation approaches for extracting general knowledge from a teacher network to guide a student network have been suggested. Most existing methods transfer knowledge from the teacher to the student network by feeding a sequence of random minibatches sampled uniformly from the data. We argue that, instead, a compact student network should be guided gradually using samples ordered in a meaningful sequence. Thus, the gap in feature representation between the teacher and student network can be bridged step by step. In this paper, we provide a curriculum learning knowledge distillation framework via instance-level sequence learning. It employs the student network of the early epoch as a snapshot to create a curriculum for the student network's next training phase. We performed extensive experiments using the CIFAR-10, CIFAR-100, SVHN, and CINIC-10 datasets. When compared with several state-of-the-art methods, our framework achieved the best performance with fewer iterations. (C) 2021 Elsevier B.V. All rights reserved.

关键词： neural networks compression Knowledge distillation Computer vision Deep learning

来源：评论

学校读者我要写书评

暂无评论

Protection of Superconducting Industrial Machinery Using RNN-Based Anomaly Detection for Implementation in Smart Sensor

引用

SENSORS 2018年第11期18卷 3933-3933页

作者： Wielgosz, Maciej Skoczen, Andrzej De Matteis, Ernesto AGH Univ Sci & Technol Fac Comp Sci Elect & Telecommun Al Adama Mickiewicza 30 PL-30059 Krakow Poland Acad Comp Ctr CYFRONET AGH Ul Nawojki 11 PL-30072 Krakow Poland AGH Univ Sci & Technol Fac Phys & Appl Comp Sci Al Adama Mickiewicza 30 PL-30059 Krakow Poland CERN European Org Nucl Res CH-1211 Geneva 23 Switzerland

Sensing the voltage developed over a superconducting object is very important in order to make superconducting installation safe. An increase in the resistive part of this voltage (quench) can lead to significant deterioration or even to the destruction of the superconducting device. Therefore, detection of anomalies in time series of this voltage is mandatory for reliable operation of superconducting machines. The largest superconducting installation in the world is the main subsystem of the Large Hadron Collider (LHC) accelerator. Therefore a protection system was built around superconducting magnets. Currently, the solutions used in protection equipment at the LHC are based on a set of hand-crafted custom rules. They were proved to work effectively in a range of applications such as quench detection. However, these approaches lack scalability and require laborious manual adjustment of working parameters. The presented work explores the possibility of using the embedded Recurrent neural Network as a part of a protection device. Such an approach can scale with the number of devices and signals in the system, and potentially can be automatically configured to given superconducting magnet working conditions and available data. In the course of the experiments, it was shown that the model using Gated Recurrent Units (GRU) comprising of two layers with 64 and 32 cells achieves 0.93 accuracy for anomaly/non-anomaly classification, when employing custom data compression scheme. Furthermore, the compression of proposed module was tested, and showed that the memory footprint can be reduced four times with almost no performance loss, making it suitable for hardware implementation.

关键词： anomaly detection recurrent neural networks neural networks compression LHC

来源：评论

学校读者我要写书评

暂无评论

Quantized neural networks: training neural networks with low precision weights and activations

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2017年第1期18卷

作者： Itay Hubara Matthieu Courbariaux Daniel Soudry Ran El-Yaniv Yoshua Bengio Department of Electrical Engineering Technion - Israel Institute of Technology Haifa Israel Department of Computer Science and Department of Statistics Université de Montréal Montréal Canada Department of Statistics Columbia University New York Department of Computer Science Technion - Israel Institute of Technology Haifa Israel

We introduce a method to train Quantized neural networks (QNNs) -- neural networks with extremely low precision (e.g., 1-bit) weights and activations, at run-time. At traintime the quantized weights and activations are used for computing the parameter gradients. During the forward pass, QNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations. As a result, power consumption is expected to be drastically reduced. We trained QNNs over the MNIST, CIFAR-10, SVHN and ImageNet datasets. The resulting QNNs achieve prediction accuracy comparable to their 32-bit counterparts. For example, our quantized version of AlexNet with 1-bit weights and 2-bit activations achieves 51% top-1 accuracy. Moreover, we quantize the parameter gradients to 6-bits as well which enables gradients computation using only bit-wise operation. Quantized recurrent neural networks were tested over the Penn Treebank dataset, and achieved comparable accuracy as their 32-bit counterparts using only 4-bits. Last but not least, we programmed a binary matrix multiplication GPU kernel with which it is possible to run our MNIST QNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The QNN code is available online.

关键词： computer vision deep learning energy efficient neural networks language models neural networks compression

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：