检索结果-内蒙古大学图书馆

A reconfigurable multi-precision quantization-aware nonlinear activation function hardware module for DNNs

MICROELECTRONICS JOURNAL 2024年 151卷

作者： Hong, Qi Liu, Zhiming Long, Qiang Tong, Hao Zhang, Tianxu Zhu, Xiaowen Zhao, Yunong Ru, Hua Zha, Yuxing Zhou, Ziyuan Wu, Jiashun Tan, Hongtao Hong, Weiqiang Xu, Yaohua Guo, Xiaohui Anhui Univ Sch Integrated Circuits Key Lab Intelligent Comp & Signal Proc Minist Educ Hefei 230601 Peoples R China Huadong Photoelectron IC Inst Bengbu 233030 Anhui Peoples R China Dalian Univ Technol State Key Lab High Performance Precis Mfg Dalian 116024 Peoples R China Dalian Univ Technol Key Lab Micro Nano Technol & Syst Liaoning Prov Dalian 116024 Peoples R China

In recent years, the increasing variety of nonlinear activation functions (NAFs) in deep neural networks (DNNs) has led to higher computational demands. However, hardware implementation faces challenges such as lack of flexibility, high hardware cost, and limited accuracy. This paper proposes a highly flexible and low-cost hardware solution for implementing activation functions to overcome these issues. Based on the piecewise linear (PWL) approximation method, our method supports NAFs with different accuracy configurations through a customized implementation strategy to meet the requirements in different scenario applications. In this paper, the symmetry of the activation function is investigated, and incorporate curve translation preprocessing and data quantization to significantly reduce hardware storage costs. The modular hardware architecture proposed in this study supports NAFs of multiple accuracies, which is suitable for designing deep learning neural network accelerators in various scenarios, avoiding the need to design dedicated hardware circuits for the activation function layer and enhances circuit design efficiency. The proposed hardware architecture is validated on the Xilinx XC7Z010 development board. The experimental results show that the average absolute error (AAE) is reduced by about 35.6 % at a clock frequency of 312.5 MHz. Additionally, the accuracy loss of the model is maximized to-0.684 % after replacing the activation layer function of DNNs under the PyTorch framework.

关键词： Nonlinear activation functions Deep neural networks piecewise linear approximation Hardware architecture Customized floating-point quantization

来源：评论

学校读者我要写书评

暂无评论

Utilize the shift-and-add architecture to approximate multiple nonlinear functions

引用

INTERNATIONAL JOURNAL OF CIRCUIT THEORY AND APPLICATIONS 2021年第7期49卷 2290-2297页

作者： Xiong, Botao Sui, Yuanfeng Jia, Zhi Li, Sicun Chang, Yuchun Dalian Univ Technol Sch Microelect Dalian 116024 Liaoning Peoples R China

The piecewise linear approximation method is used to estimate the non-linear functions, which is implemented by the shift-and-add architecture consisting of three blocks: "adder tree," "multiplexer," and "segment index encoder." Compared with state of the art, the design flow in this letter enables the "adder tree" and "multiplexer" blocks can be shared by multiple nonlinear functions. In other words, the shift-and-add architecture can approximate multiple nonlinear functions with a slight overhead. The fused Log and Antilog converter demonstrates the feasibility of the proposed method. The Antilog function is implemented by the Log converter at the cost of 14% cell area and 6% delay.

关键词： fused log and antilog converter logarithmic number system piecewise linear approximation shift‐ and‐ add architecture

来源：评论

学校读者我要写书评

暂无评论

Signal Source Distribution approximation to Speedup Scalar Quantizer Design

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2021年 69卷 6314-6328页

作者： Anavangot, Vijay Kumar, Animesh Indian Inst Technol Dept Elect Engn Mumbai 400076 Maharashtra India

Classical quantizer design approaches using the Lloyd-Max algorithm (or k-means) have served signal processing applications for more than three decades. With the advent of distributed signal processing and machine learning at edge devices, novel alternatives for quantizers design will be desired to address the energy, communication and hardware constraints. To address these resource challenges, we propose a model-driven approach, termed Approximate Lloyd-Max (ALM) design, based on piecewise linear approximation of the signal-source probability density. From the principles of the ALM design, we develop a data-driven quantizer, or Learning ALM (LALM), using statistical learning methods. By mathematical analysis, we show convergence of the ALM quantizer near the limit of the Lloyd-Max quantizer. Both ALM and LALM quantizers satisfy asymptotic optimality and exponential convergence rate. Simulation performed over smooth signal source distributions validate our mathematical analysis. Experiments for LALM quantizer are implemented on an Android-based edge device, and the proposed quantizer demonstrate improved performance over k-means, in terms of algorithm speedup, energy usage and memory utilization.

关键词： Quantization (signal) piecewise linear approximation probability distribution computational efficiency

来源：评论

学校读者我要写书评

暂无评论

piecewise linear Plus Quadratic Surrogate Model for Real-Time Optimization ⁎

引用

IFAC-PapersOnLine 2024年第14期58卷 556-562页

作者： Duo Zhang Xiang Li Kody Kazda Zhijiang Shao College of Control Science and Engineering Zhejiang University Hangzhou 310027 China Department of Chemical Engineering Queen’s University Kington ON K7L 3N6 Canada

Surrogate models are important in real-time optimization (RTO) when the first-principles model is unavailable or computationally challenging for online optimization. Among different surrogate models, the continuous piecewise linear (CPWL) model enjoys the universal approximation ability and potential computational benefits. However, the CPWL surrogate model poses three challenges to current RTO algorithms. First, the solution of a CPWL model is always located on the boundary of a polytopic subregion, while the plant optimum may be in the interior of a subregion. Second, the CPWL model is nonsmooth, which cannot be handled by RTO methods that rely on gradient matching. Third, the resulting nonsmooth optimization subproblems are hard to solve. This paper addresses the difficulties by adding a quadratic function to the CPWL surrogate model, extending a classical RTO method to accommodate nonsmoothness, and exploiting the diffierence-of-convex structure of the surrogate model for Efficient solution. The advantages of the proposed method are demonstrated through a benchmark problem in the RTO literature.

关键词： real-time optimization piecewise linear approximation surrogate modifier adaptation difference-of-convex

来源：评论

学校读者我要写书评

暂无评论

System optimal routing of traffic flows with user constraints using linear programming

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2021年第3期293卷 863-879页

作者： Angelelli, E. Morandi, V Savelsbergh, M. Speranza, M. G. Univ Brescia Dept Econ & Management Brescia Italy Free Univ Bozen Bolzano Fac Sci & Technol Bolzano Italy Georgia Inst Technol H Milton Stewart Sch Ind & Syst Engn Atlanta GA 30332 USA

For static traffic assignment problems, it is well known that (1) for some users the experienced travel time in a system optimum assignment can be substantially higher than the experienced travel time in a user equilibrium assignment, and (2) the total travel time in user equilibrium can be substantially higher than the total travel time in system optimum. By seeking system optimal traffic flows subject to user constraints, a compromise assignment can be obtained that balances system and user objectives. To this aim, a linear model and an efficient heuristic algorithm are proposed in this paper. A computational study shows that the proposed model, along with the heuristic algorithm, is able to provide fair solutions with near-optimal total travel time within very short computational time. (C) 2021 The Authors. Published by Elsevier B.V.

关键词： Traffic Congestion Latency function Constrained system optimum linear programming piecewise linear approximation

来源：评论

学校读者我要写书评

暂无评论

Hybrid simplicial-randomized approximate stochastic dynamic programming for multireservoir optimization

引用

COMPUTATIONAL MANAGEMENT SCIENCE 2024年第1期21卷 31-31页

作者： Zephyr, Luckny Lamond, Bernard F. Lang, Pascal Laurentian Univ Sch Business Adm Barthi Sch Engn & Comp Sci Cross Appointment Sudbury ON P3E 2C6 Canada Univ Laval Dept Operat & Syst Decis Quebec City PQ G1V 0A6 Canada

We revisit an approximate stochastic dynamic programming method that we proposed earlier for the optimization of multireservoir problems. The method exploited the convexity properties of the value function to sample the reservoir level space based on the local curvature of the value function, which is estimated by the difference between a lower and an upper bounds (error bound). Unlike the previous approach in which the state space is exhaustively partitioned into full dimensional simplices whose vertices formed a discrete grid over which the value function was approximated, here we propose instead a new randomized approach for selecting the grid points from a small number of randomly sampled simplices from which an error bound is estimated. Results of numerical experiments on three literature test problems and simulated midterm reservoir optimization problems illustrate the advantages of the randomized approach which can solve models of higher dimensions than with the exhaustive approach.

关键词： Reservoir optimization Stochastic dynamic programming Simplicial-randomized approximation piecewise linear approximation

来源：评论

学校读者我要写书评

暂无评论

On the Performance of Size-Limited piecewise Compression for WSN Systems

On the Performance of Size-Limited Piecewise Compression for...

引用

IEEE Conference on Wireless Communications and Networking

作者： Sanku Kumar Roy Ioanis Nikolaidis Department of Computing Science University of Alberta Canada

ISBN: (数字)9798350368369

ISBN: (纸本)9798350368376

Certain lossy compression schemes use a polynomial approximation approach to represent a time series of data. It is common knowledge that higher degree polynomials can better approximate functions than smaller degree ones. It is also often the case that in embedded wireless sensor systems, resource constraints force us to limit the size that can be used to represent compressed data. We find that, when attempting to minimize the maximum reconstruction error by compressing into limited storage using polynomial-based compression schemes, higher degree polynomials perform poorly. By contrast, simple piecewise linear approximations with line segments whose endpoints are data points of the time series, and subject to the same storage limitations, are better performing in terms of reconstruction error.

关键词： Wireless communication Wireless sensor networks Heuristic algorithms Time series analysis piecewise linear approximation Force approximation algorithms Polynomials Sensor systems Dynamic programming

来源：评论

学校读者我要写书评

暂无评论

On Relationship of Multilayer Perceptrons and piecewise Polynomial Approximators

引用

IEEE SIGNAL PROCESSING LETTERS 2021年 28卷 1813-1817页

作者： Lin, Ruiyuan You, Suya Rao, Raghuveer Kuo, C-C Jay Univ Southern Calif Ming Hsieh Dept Elect & Comp Engn Los Angeles CA 90089 USA Army Res Lab Adelphi MD 20783 USA

The relationship between a multilayer perceptron (MLP) regressor and a piecewise polynomial approximator is investigated in this work. We propose an MLP construction method, including the choice of activation, the specification of neuron numbers and filter weights. Through the construction, a one-to-one correspondence between an MLP and a piecewise polynomial is established. Especially, we point out that the form of nonlinear activation is related to the polynomial order. Since the approximation capability of piecewise polynomials is well understood, our study sheds new light on the universal approximation capability of an MLP.

关键词： Neurons Multilayer perceptrons Tools piecewise linear approximation Biological neural networks Transforms Taylor series Multilayer perceptron feedforward neural network approximation theory piecewise polynomial approximation

来源：评论

学校读者我要写书评

暂无评论

Analysis of strategic bidding of a DER aggregator in energy markets through the Stackelberg game model with the mixed-integer lower-level problem

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2023年第1期152卷

作者： Lei, Zhenxing Liu, Mingbo Shen, Zhijun South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China South China Univ Technol Guangdong Key Lab Clean Energy Technol Guangzhou 510640 Peoples R China Elect Power Dispatching Control Ctr Guangdong Grid Guangzhou 510600 Peoples R China

An open electricity market environment provides sufficient conditions for a distributed energy resource (DER) aggregator to participate in the day-ahead electricity market. Therefore, a one-leader multi-follower Stackelberg game model is developed to analyze the strategic bidding behavior of a DER aggregator and one of the lower -level problems is to check the security of the distribution system (DS), which is elaborated by the mixed -integer second-order cone programming (MISOCP) model considering the active power loss and discrete con-trols in the DS. Moreover, we first use the Karush-Kuhn-Tucker (KKT)-based reformulation approach to trans-form the model into a one-leader one-follower model. However, the objective function of the upper-level model cannot be completely linearized by using the strong dual theorem because the active power loss of the DS is considered in the lower-level security check problem. Therefore, we propose a piecewise linear approximation approach based on the data-driven algorithm to linearize the upper-level objective function. Moreover, we apply multivariate linear surrogation to transform this one-leader one-follower model into a single-level MISOCP model. Finally, we verify the correctness of the proposed algorithm by comparing it with the particle swarm optimization (PSO) algorithm in a small integrated transmission and distribution (T&D) system and validate the effectiveness of the proposed algorithm in a practical integrated T&D system.

关键词： Distributed energy resource aggregator Strategic bidding Mixed-integer second-order cone programming Security check piecewise linear approximation Multivariate linear surrogation

来源：评论

学校读者我要写书评

暂无评论

Physics Informed piecewise linear Neural Networks for Process Optimization

引用

COMPUTERS & CHEMICAL ENGINEERING 2023年第1期174卷

作者： Koksal, Ece Serenat Aydin, Erdal Koc Univ Dept Chem & Biol Engn TR-34450 Istanbul Turkiye Koc Univ TUPRAS Energy Ctr KUTEM TR-34450 Istanbul Turkiye

Constructing first-principles models is usually a challenging and time-consuming task due to the complexity of real-life processes. On the other hand, data-driven modeling, particularly a neural network model, often suffers from overfitting and lack of useful and high-quality data. At the same time, embedding trained machine learning models directly into the optimization problems has become an effective and state-of-the-art approach for sur-rogate optimization, whose performance can be improved by physics-informed machine learning. This study proposes using piecewise linear neural network models with physics-informed knowledge for optimization problems with neural network models embedded. In addition to using widely accepted and naturally piecewise linear rectified linear unit (ReLU) activation functions, this study also suggests piecewise linear approximations for the hyperbolic tangent activation function to widen the domain. Optimization of three case studies, a blending process, an industrial distillation column, and a crude oil column are investigated. Physics-informed trained neural network-based optimal results are closer to global optimality for all cases. Finally, associated CPU times for the optimization problems are much shorter than the standard optimization results.

关键词： Machine learning artificial neural networks physics-informed neural networks piecewise linear approximation mixed integer linear programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：