Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer f...
详细信息
Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer feedforward neural networks (SLFNs) is online sequential extreme learning machine (OS-ELM), which can learn data one-by-one or chunk-by-chunk at fixed or varying sizes. It is based on the ideas of extreme learning machine (ELM), in which the input weights and hidden layer biases are randomly chosen and then the output weights are determined by the pseudo-inverse operation. The learning speed of this algorithm is extremely high. However, it is not good to yield generalization models for noisy data and is difficult to initialize parameters in order to avoid singular and ill-posed problems. In this paper, we propose an improvement of OS-ELM based on the bi-objective optimization approach. It tries to minimize the empirical error and obtain small norm of network weight vector. Singular and ill-posed problems can be overcome by using the Tikhonov regularization. This approach is also able to learn data one-by-one or chunk-by-chunk. Experimental results show the better generalization performance of the proposed approach on benchmark datasets. (C) 2011 Elsevier B.V. All rights reserved.
This paper applies artificial neural networks (ANNs) trained with a multiobjective algorithm to preprocess the ground penetrating radar data obtained from a finite-difference time-domain (FDTD) model. This preprocessi...
详细信息
This paper applies artificial neural networks (ANNs) trained with a multiobjective algorithm to preprocess the ground penetrating radar data obtained from a finite-difference time-domain (FDTD) model. This preprocessing aims at improving the target's reflected wave signal-to-noise ratio (SNR). Once trained, the NN behaves as an adaptive filter which minimizes the cross-validation error. Results considering both white and colored Gaussian noise, with many different SNR, are presented and they show the effectiveness of the proposed approach.
This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that super...
详细信息
This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.
This paper applies a Neural Networks (NN) multiobjective learning algorithm called the Minimum Gradient Method (MGM) to filter noise in regression problems. This method is based on the concept that the learning is a b...
详细信息
ISBN:
(纸本)9780769534404
This paper applies a Neural Networks (NN) multiobjective learning algorithm called the Minimum Gradient Method (MGM) to filter noise in regression problems. This method is based on the concept that the learning is a bi-objective problem aiming at minimizing the empirical risk (training error) and the function complexity. The complexity is modeled as the norm of the network output gradient. After training, the NN behaves as an adaptive filter which minimizes the cross-validation error. The NN trained with this method can be used to pre-process the data and help reduce the signal-to-noise ratio (SNR). Some results are presented and they show the effectiveness of the proposed approach.
暂无评论