A improved gradient-based backpropagation training method is proposed for neural networks in this paper. Based on the Barzilai and Borwein steplength update and some technique of Resilient Propagation method, we adapt...
详细信息
A improved gradient-based backpropagation training method is proposed for neural networks in this paper. Based on the Barzilai and Borwein steplength update and some technique of Resilient Propagation method, we adapt the new learning rate to improves the speed and the success rate. Experimental results show that the proposed method has considerably improved convergence speed, and for the chosen test problems, outperforms other well-known training methods.
Slower convergence and longer training times are the disadvantages often mentioned when the conventional backpropagation (BP) algorithm are compared with other competing techniques. In addition, in the conventional BP...
详细信息
Slower convergence and longer training times are the disadvantages often mentioned when the conventional backpropagation (BP) algorithm are compared with other competing techniques. In addition, in the conventional BP algorithm, the learning rate is fixed and that it is uniform for all the weights in a layer. In this paper, we propose an efficient acceleration technique, the backpropagation with adaptive learning rate and momentum term, which is based on the conventional BP algorithm by employing an adaptive learning rate and momentum factor, where the learning rate and momentum rate are adjusted at each iteration to reduce the training time. Simulation results indicate a superior convergence speed as compared to other competing methods.
Neural networks have traditionally been applied to recognition problems, and most learning algorithms are tailored to those problems. The authors discuss the requirements of learning for generalization, which is NP-co...
详细信息
Neural networks have traditionally been applied to recognition problems, and most learning algorithms are tailored to those problems. The authors discuss the requirements of learning for generalization, which is NP-complete and cannot be approached by traditional methods based on gradient descent. They present a stochastic learning algorithm based on simulated annealing in weight space. The convergence properties and feasibility of the algorithm are verified.< >
The problem of correctly evaluating noisy and incorrect data for the interpretation of ultrasonic sensor signals is addressed. Neural networks, with their inherent characteristics of adaptivity and high fault and nois...
详细信息
The problem of correctly evaluating noisy and incorrect data for the interpretation of ultrasonic sensor signals is addressed. Neural networks, with their inherent characteristics of adaptivity and high fault and noise tolerance, are well suited for such tasks. A backpropagation algorithm is described for the control of the tracking behavior of an autonomous mobile robot. Input data are provided by three ultrasonic sensors mounted on the front of the vehicle. For more flexibility the behavior and learning capability of the tracking algorithm have been improved using different networks.< >
For the learning process of multilayer feedforward neural network, the constructive approach of Modified backpropagation algorithm (MBP), with optimum initialization is proposed. One of the common complaints about the...
详细信息
For the learning process of multilayer feedforward neural network, the constructive approach of Modified backpropagation algorithm (MBP), with optimum initialization is proposed. One of the common complaints about the Standard backpropagation algorithm (SBP) is that it is very slow. Even simple problems may take hundreds of iterations to converge. SBP algorithm reduces only non-linear errors. Much work, therefore has been done in search of faster methods. One of such approach is modified form of the Standard backpropagation algorithm. Modified backpropagation algorithm consists of minimizing the sum of the squares of linear and non- linear errors for all output units. This leads to an efficient process in the network. Proper initialization always plays a key role in the robust neural networks. Therefore, the optimum initialization method is used for weight initialization, which ensures the outputs of neurons are in the active region and the range of activation function is fully utilized. Since the proposed method uses the constructive approach, there is no need to make a prior estimate of the correct network size. The proposed method is implemented on 2 bit parity problem, 4 bit parity checker and encoder problem and produced good results.
In recent years much effort has been spent trying to develop more efficient variations of the backpropagation learning algorithm. This has led to a combinatorial explosion of learning methods of which no detailed eval...
详细信息
In recent years much effort has been spent trying to develop more efficient variations of the backpropagation learning algorithm. This has led to a combinatorial explosion of learning methods of which no detailed evaluation exists. We have analyzed the most important algorithms and extracted their minimal building blocks. By arranging these building blocks in different forms, and testing the resulting algorithms, we obtained new combinations which were benchmarked in a commercial workstation. Our results show which factors are responsible for the increased speed-up of the tested algorithms. These results could lead to better learning methods for neural networks.
A distributed backpropagation algorithm for a fully connected multilayered neural network on a distributed-memory multiprocessor system is presented. The neurons on each layer are partitioned into p disjoint sets, and...
详细信息
A distributed backpropagation algorithm for a fully connected multilayered neural network on a distributed-memory multiprocessor system is presented. The neurons on each layer are partitioned into p disjoint sets, and each set is mapped on a processor of a p-processor system. The algorithm, the communication pattern among the processors, and their time/space complexities are investigated, and the theoretical upper bound on speedup is obtained. The experimental speedup obtained with the algorithm on a ring of 32 transputers, which confirms the model and analysis, is reported. It is found that the choice of processor interconnection topology does not influence the speedup ratio.< >
The paper describes the implementation of a systolic array for a multilayer perceptron on ALTERA FLEX10KE FPGAs with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algori...
详细信息
The paper describes the implementation of a systolic array for a multilayer perceptron on ALTERA FLEX10KE FPGAs with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algorithm is shown. It better exploits the parallelism because both the forward and backward phases can be performed simultaneously. As a result, a combined systolic array structure is proposed for both phases. Analytic expressions show that the pipelined version is more efficient than the non-pipelined version. The design is implemented and simulated using VHDL at different levels of abstraction and finally mapped on FPGAs.
The authors propose a fuzzy logic controlled implementation of the backpropagation training algorithm for layered perceptrons. The heuristics for adjusting the value of the learning rate eta are incorporated into a si...
详细信息
The authors propose a fuzzy logic controlled implementation of the backpropagation training algorithm for layered perceptrons. The heuristics for adjusting the value of the learning rate eta are incorporated into a simple fuzzy control system. This provides automatic tuning of the learning rate parameter depending on the shape of the error surface. The application of this straightforward procedure was shown to be able to dramatically improve training time in some problems.< >
暂无评论