检索结果-内蒙古大学图书馆

Contrast enhancement for backpropagation

IEEE TRANSACTIONS ON NEURAL NETWORKS 1996年第2期7卷 515-524页

作者： Kwon, TM Cheng, H Department of Electrical and Computer Engineering University of Minnesota Duluth Duluth MN USA

This paper analyzes the effect of data-contrast to a backpropagation (BP) network and introduces a data preprocessing algorithm that can improve the efficiency of the standard BP learning. The basic idea is to transform input data to a range that associates the high-slope region of the sigmoid function where a relatively large modification of weights occurs. A simple uniform transformation to such a desired range, however, can lead to a slow and unbalanced learning if the data distribution is heavily skewed. To facilitate data processing on such distribution, we propose a modified histogram equalization technique which enhances the spacing between the data points in the heavily concentrated regions of the distribution.

关键词： Algorithm design and analysis backpropagation algorithms Histograms Data processing Training data Error correction Shape Neurons

来源：评论

学校读者我要写书评

暂无评论

30 YEARS OF ADAPTIVE NEURAL NETWORKS - PERCEPTRON, MADALINE, AND backpropagation

引用

PROCEEDINGS OF THE IEEE 1990年第9期78卷 1415-1442页

作者： WIDROW, B LEHR, MA Information Systems Laboratory Department of Electrical Engineering University of Stanford Stanford CA USA

Fundamental developments in feedforward artificial neural networks from the past thirty years are reviewed. The history, origination, operating characteristics, and basic theory of several supervised neural-network training algorithms (including the perceptron rule, the least-mean-square algorithm, three Madaline rules, and the backpropagation technique) are described. The concept underlying these iterative adaptation algorithms is the minimal disturbance principle, which suggests that during training it is advisable to inject new information into a network in a manner that disturbs stored information to the smallest extent possible. The two principal kinds of online rules that have developed for altering the weights of a network are examined for both single-threshold elements and multielement networks. They are error-correction rules, which alter the weights of a network to correct error in the output response to the present input pattern, and gradient rules, which alter the weights of a network during each pattern presentation by gradient descent with the objective of reducing mean-square error (averaged over all training patterns).< >

关键词： Adaptive systems Neural networks Subspace constraints Artificial neural networks backpropagation algorithms History Least squares approximation Biological system modeling Machine learning Pattern recognition

来源：评论

学校读者我要写书评

暂无评论

The Symplectic Adjoint Method: Memory-Efficient backpropagation of Neural-Network-Based Differential Equations

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024年第8期35卷 10526-10538页

作者： Matsubara, Takashi Miyatake, Yuto Yaguchi, Takaharu Osaka Univ Grad Sch Engn Sci Osaka 5608531 Japan Osaka Univ Cybermedia Ctr Osaka 5608531 Japan Kobe Univ Grad Sch Syst Informat Kobe Hyogo 6578501 Japan

The combination of neural networks and numerical integration can provide highly accurate models of continuous-time dynamical systems and probabilistic distributions. However, if a neural network is used $\bm{n}$ times during numerical integration, the whole computation graph can be considered as a network $\bm{n}$ times deeper than the original. The backpropagation algorithm consumes memory in proportion to the number of uses times of the network size, causing practical difficulties. This is true even if a checkpointing scheme divides the computation graph into subgraphs. Alternatively, the adjoint method obtains a gradient by a numerical integration backward in time;although this method consumes memory only for single-network use, the computational cost of suppressing numerical errors is high. The symplectic adjoint method proposed in this study, an adjoint method solved by a symplectic integrator, obtains the exact gradient (up to rounding error) with memory proportional to the number of uses plus the network size. The theoretical analysis shows that it consumes much less memory than the naive backpropagation algorithm and checkpointing schemes. The experiments verify the theory, and they also demonstrate that the symplectic adjoint method is faster than the adjoint method and is more robust to rounding errors.

关键词： Neural networks Artificial neural networks backpropagation Memory management Computational efficiency Checkpointing backpropagation algorithms Adjoint method backpropagation checkpointing scheme neural networks ordinary differential equation (ODE) symplectic integrator

来源：评论

学校读者我要写书评

暂无评论

AN ADAPTIVE STEP-SIZE FOR backpropagation USING LINEAR LOWER BOUNDING FUNCTIONS

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 1995年第5期43卷 1243-1248页

作者： YU, M CHANG, TS Department of Electrical and Computer Engineering University of California Davis CA USA

An adaptive step size is presented for the backpropagation algorithm in feedforward neural nets using linear lower bounding functions, Basically, a linear lower bounding function (LLBF) for a given function over an interval is a linear function that lies below the given function and matches the original function value at one end point, To search for an adaptive step size, an LLBF for the error function, which is expressed in terms of the step size, is derived, Since the error in a neural net can never be smaller than zero, it is plausible not to take a step larger than the step size when the associated LLBF reaches zero, In this paper, an adaptive learning algorithm based on the above idea is given, Numerical examples are used to illustrate its feasibility and to compare it with some previous results.

关键词： backpropagation algorithms Neural networks Convergence Feedforward neural networks Adaptive algorithm Signal processing algorithms

来源：评论

学校读者我要写书评

暂无评论

IMPROVING GENERALIZATION PERFORMANCE USING DOUBLE backpropagation

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS 1992年第6期3卷 991-997页

作者： DRUCKER, H LECUN, Y MONMOUTH COLL LONG BRANCH NJ 07764 USA AT&T BELL LABS HOLMDEL NJ 07733 USA

In order to generalize from a training set to a test set, it is desirable that small changes in the input space of a pattern do not change the output components. This can be done by including variations of the input space as part of the training set-but this is computationally very expensive. Another method is to force this behavior as part of the training algorithm. This is done in double backpropagation by forming an energy function that is the sum of the normal energy term found in backpropagation and an additional term that is a function of the Jacobian. Significant improvement is shown with different architectures and different test sets, especially with architectures that had previously been shown to have very good performance when trained using backpropagation. It also will be shown that double backpropagation, as compared to backpropagation, creates weights that are smaller thereby causing the output of the neurons to spend more time in the linear region.

关键词： Testing backpropagation algorithms Jacobian matrices Neurons Signal to noise ratio Neural networks

来源：评论

学校读者我要写书评

暂无评论

TOLERANCE TO ANALOG HARDWARE OF ON-CHIP LEARNING IN backpropagation NETWORKS

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS 1995年第5期6卷 1045-1052页

作者： DOLENKO, BK CARD, HC UNIV MANITOBA WINNIPEGMB R3T 2N2CANADA

When training an artificial neural network using the popular backpropagation algorithm, implementation in dedicated analog hardware offers an attractive alternative for reasons of speed, compactness, and the lack of limited resolution in computation that is present in digital hardware, Despite these advantages, until now the use of analog hardware in backpropagation networks has been primarily limited to the forward computation, The learning computations are usually performed off-chip, because it is not immediately evident that the nonidealities inherent in analog hardware allow adequate learning to take place, In this paper we present results of simulations performed assuming both forward and backward computation are done on-chip using analog components, Aspects of analog hardware studied are component variability (variability in multiplier gains and zero offsets),limited voltage ranges, components (multipliers) that only approximate the computations in the backpropagation algorithm, and capacitive weight decay, It is shown that backpropagation networks can learn to compensate for all these shortcomings of analog circuits except for zero offsets, and the latter are correctable with minor circuit complications, Variability in multiplier gains is not a problem, and learning is still possible despite limited voltage ranges and function approximations, Fixed component variation from fabrication is shown to be less detrimental to learning than component variation due to noise, Weight decay is tolerable provided it is sufficiently:small, which implies frequent refreshing by rehearsal on the training data or modest cooling of the circuits, The former approach allows for learning nonstationary problem sets.

关键词： Hardware Analog computers Voltage Function approximation Computational modeling backpropagation algorithms Analog circuits Fabrication Circuit noise Training data

来源：评论

学校读者我要写书评

暂无评论

CAN backpropagation ERROR SURFACE NOT HAVE LOCAL MINIMA

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS 1992年第6期3卷 1019-1021页

作者： YU, XH Communication Lab Department of Radio Engineering South-East University Nanjing Jiangsu China

From a theoretic point of view, we show that for an arbitrary T-element training set with t(t less-than-or-equal-to T) different inputs, the backpropagation error surface does not have suboptimal local minima if the network is capable of exactly implementing an arbitrary training set consisting of t different patterns. As a special case, the error surface of a backpropagation network with one hidden layer and t - 1 hidden units has no local minima, if the network is trained by an arbitrary T-element set with t different inputs.

关键词： Neural networks backpropagation algorithms Convergence Surface treatment Vectors Combinatorial mathematics

来源：评论

学校读者我要写书评

暂无评论

STATISTICAL-ANALYSIS OF THE SINGLE-LAYER backpropagation ALGORITHM .1. MEAN WEIGHT BEHAVIOR

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 1993年第2期41卷 573-582页

作者： BERSHAD, NJ SHYNK, JJ FEINTUCH, PL UNIV CALIF SANTA BARBARA CTR INFORMAT PROC RESDEPT ELECTR & COMP ENGNSANTA BARBARACA 93106 HUGHES AIRCRAFT CO FULLERTONCA 92634

The single-layer backpropagation algorithm is a gradient-descent method that adjusts the connection weights of a single-layer perceptron to minimize the mean-square error at the output. It is similar to the standard least mean square algorithm, except the output of the linear combiner contains a differentiable nonlinearity. In this paper, we present a statistical analysis of the mean weight behavior of the single-layer backpropagation algorithm for Gaussian input signals. It is based on a nonlinear system identification model of the desired response which is capable of generating an arbitrary hyperplane decision boundary. It is demonstrated that, although the weights grow unbounded, the algorithm, on average, quickly learns the correct hyperplane associated with the system identification model.

关键词： Statistical analysis backpropagation algorithms Algorithm design and analysis Mean square error methods Nonlinear systems System identification

来源：评论

学校读者我要写书评

暂无评论

A Digital Signage Audience Classification Model Based on the Huff Model and backpropagation Neural Network

引用

IEEE ACCESS 2020年 8卷 71708-71720页

作者： Zhang, Xun Xie, Xiaolan Wang, Yuxue Zhang, Xiaohu Jiang, Dong Yu, Chongchong Liang, Yike Beijing Technol & Business Univ Sch Comp & Informat Engn Beijing Key Lab Big Data Technol Food Safety Beijing 100048 Peoples R China Nanjing Agr Univ Coll Agr Nanjing 210095 Peoples R China Chinese Acad Sci Inst Geog Sci & Nat Resources Res IGSNRR Beijing 100101 Peoples R China

Digital signage is an important outdoor advertising medium in cities. However, advertising on digital signage often lacks pertinence. Thus, it is important to introduce an accurate digital signage audience classification method to facilitate targeted advertising. In this study, a multi-label classification model based on a backpropagation (BP) neural network and the Huff model, referred to as the Huff-BP model, is proposed to investigate digital signage audience classification. A case study is performed on outdoor digital signage within the 6th Ring Road in Beijing, China, and economic census, population census, average housing price, social media check-in and the centrality of traffic networks as research data. The data are divided into ,000 m normal grids. Multi-label classification modelling factors for various grid scales are constructed. The BP neural network classification algorithm is improved to solve the multi-label classification problem. In addition, an improved Huff model is used to calculate the digital signage influence values between each grid cell and integrated into the improved BP neural network to classify modelling factors at various scales. Finally, four metrics are used to examine the effectiveness of the proposed model. The results show that the Huff-BP-based multi-label classification model achieves relatively good classification results, and the digital signage audiences are mainly concentrated within the 4th Ring Road and near the 5th Ring Road.

关键词： Classification algorithms Neural networks Prediction algorithms Roads Data models backpropagation Advertising Digital signage backpropagation algorithms classification algorithms geographic information systems

来源：评论

学校读者我要写书评

暂无评论

DYNAMIC backpropagation ALGORITHM FOR NEURAL NETWORK CONTROLLED RESONATOR-BANK ARCHITECTURE

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-ANALOG AND DIGITAL SIGNAL PROCESSING 1992年第2期39卷 99-108页

作者： SZTIPANOVITS, J VANDERBILT UNIV DEPT ELECT ENGNNASHVILLETN 37235 USA

This paper presents an adaptive processing system that consists of a resonator-based digital filter and a neural network. The filter section realizes the dynamics of the adaptive system, while the transfer characteristics are controlled by the neural network. The paper focuses on on-line training algorithms that can create an association between features of the input signal of the neural network and dynamic responses of the digital filter. A dynamic back propagation algorithm is derived for training the network in closed-loop configurations, when a feedback path exists between the output of the digital filter section and inputs to the neural network. Simulations results show that the neural network controlled resonator-bank architecture is computationally feasible and can be used as a general building block in a wide range of identification and control problems.

关键词： backpropagation algorithms Neural networks Digital filters Adaptive systems Adaptive filters Resonator filters Programmable control Adaptive control Control systems Heuristic algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：