It is difficult for Extreme Learning Machine (ELM) to estimate the number of hidden nodes used to match with the learning data. In this paper, a novel pruning algorithm based on sensitivity analysis is proposed for EL...
详细信息
ISBN:
(纸本)9783642412783;9783642412776
It is difficult for Extreme Learning Machine (ELM) to estimate the number of hidden nodes used to match with the learning data. In this paper, a novel pruning algorithm based on sensitivity analysis is proposed for ELM. The measure to estimate the necessary number of hidden layer nodes is presented according to the defined sensitivity. When the measure is below the given threshold, the nodes with smaller sensitivities are removed from the existent network all together. Experimental results show that the proposed method can produce more compact neural network than some other existing similar algorithms.
A modified version of a formal pruning algorithm initially proposed by Englebercht [8] using variance analysis of sensitivity is presented. We propose a new modification of the algorithm by applying the pruning proced...
详细信息
ISBN:
(纸本)0780383796
A modified version of a formal pruning algorithm initially proposed by Englebercht [8] using variance analysis of sensitivity is presented. We propose a new modification of the algorithm by applying the pruning procedure on each layer starting from the output layer to the input layer. Contrarily, to the work of Englebercht where the pruning is performed on the entire net that we denote in this paper global pruning, we shall prune layer by layer with the use of a pruning decision based on a local parameter variance nullity coefficient (LPVN). These coefficients are then classified in an ordered list which allows the decision making of coefficients and neurons removal in order to get the best neural network pruned. A comparison study is given on some real world examples showing that in some cases we ca reach about 30% improvement in terms of learning and generalization.
Based on the method of Skeletonization, the concept of influence factor is introduced in this paper. A method for trimming the fat from a Back Propagation (BP) neural network is proposed by modifying weight and influe...
详细信息
ISBN:
(纸本)9781538635247
Based on the method of Skeletonization, the concept of influence factor is introduced in this paper. A method for trimming the fat from a Back Propagation (BP) neural network is proposed by modifying weight and influence factor alternately, and node with the least influence factor was deleted. This method is applied to modeling superheated steam temperature system of plant station. Simulation results show that this pruning algorithm meets the demand of precision with higher convergence rate, and the generalization ability greatly improves.
In this paper, a novel pruning algorithm is proposed for self-organizing the feed-forward neural network based on the sensitivity analysis, named novel pruning feed-forward neural network (NP-FNN). In this study, the ...
详细信息
ISBN:
(纸本)9781424435494
In this paper, a novel pruning algorithm is proposed for self-organizing the feed-forward neural network based on the sensitivity analysis, named novel pruning feed-forward neural network (NP-FNN). In this study, the number of hidden neurons is determined by the output's sensitivity to the hidden nodes. This technique determines the relevance of the hidden nodes by analyzing the Fourier decomposition of the variance. Then each hidden node can obtain a contribution ratio. The connected weights of the hidden nodes with small ratio will be set as zeros. Therefore, the computational cost of the training process will be reduced significantly. It is clearly shown that the novel pruning algorithm minimizes the complexity of the final feed-forward neural network. Finally, computer simulation results are carried out to demonstrate the effectiveness of the proposed algorithm.
Breadth-first search (BFS) is one of the most fundamental algorithms for searching a graph. In our previous work, a mobile-assisted diagnosis scheme has been proposed and we have designed a Disease-Symptom data model ...
详细信息
ISBN:
(纸本)9789811317477;9789811317460
Breadth-first search (BFS) is one of the most fundamental algorithms for searching a graph. In our previous work, a mobile-assisted diagnosis scheme has been proposed and we have designed a Disease-Symptom data model as a large knowledge base. In this work, we present the design and implementation of a pruning algorithm to capture a part of the large graph-based data model. Generally, to search a large graph, most of the graph queries are too long and very cumbersome to write and sometimes very difficult to implement. pruning algorithm is one of the prominent solutions of this problem. It results a subgraph or forest for the desired input. Here, our proposed pruning algorithm is multi-source sequential BFS algorithm and it is demonstrated on Disease-Symptom graph database.
This paper deals with a new approach to detect the structure (i.e. determination of the number of hidden units) of a feedforward neural network (FNN). This approach is based on the principle that any FNN could be repr...
详细信息
ISBN:
(纸本)0780370414
This paper deals with a new approach to detect the structure (i.e. determination of the number of hidden units) of a feedforward neural network (FNN). This approach is based on the principle that any FNN could be represented by a Volterra series such as a nonlinear input-output model. The new proposed algorithm is based on the following three steps: first, we develop the nonlinear activation function of the hidden layer's neurons in a Taylor expansion, secondly we express the neural network output as a NARX (nonlinear auto regressive with exogenous input) model and finally, by appropriately using the nonlinear order selection algorithm proposed by Kortmann-Unbehauen, we select the most relevant signals on the NARX model obtained. Starting from the output layer, this pruning procedure is performed on each node in each layer. Using this new algorithm with the standard backpropagation (SBP) and over various initial conditions, we perform Monte Carlo experiments leading to a drastic reduction in the nonsignificant network hidden layer neurons.
Shared knowledge mining refers to learn the sharing knowledge of different things, and applying the learned knowledge to the unknowns to accelerate the recognition of them. For the problems of low efficiency and the t...
详细信息
Shared knowledge mining refers to learn the sharing knowledge of different things, and applying the learned knowledge to the unknowns to accelerate the recognition of them. For the problems of low efficiency and the t...
详细信息
ISBN:
(纸本)9781538636749
Shared knowledge mining refers to learn the sharing knowledge of different things, and applying the learned knowledge to the unknowns to accelerate the recognition of them. For the problems of low efficiency and the traditional memory classification algorithm unable to deal with massive data, then combined with the cloud computing technology, A Parallel Shared Decision Tree (PSDT) algorithm had been proposed. Although this algorithm improved the efficiency, the performance still needs to be optimized for leaving the influence of the training set noise out of consideration. So, in this paper, based on the PSDT algorithm, a parallel shared decision tree imprecise error pruning (PSDT-IEP) algorithm is proposed. In our algorithm, we reduced the impact of unreliability by using the classification number of the uncertainty probability error of the data set to prune, which improves the accuracy of the algorithm. Along with the increase of the data set, the superiority of PSDT-IEP is more obvious.
To improve the prediction accuracy and reduce the computational complexity of network traffic prediction based on feed-forward neural network(FNN),the partial least squares(PLS) pruning algorithm was proposed to optim...
详细信息
ISBN:
(纸本)9781510823808
To improve the prediction accuracy and reduce the computational complexity of network traffic prediction based on feed-forward neural network(FNN),the partial least squares(PLS) pruning algorithm was proposed to optimize the network topology *** data of network traffic has the characteristics of burst,nonlinear and time variation,which results in the traditional neural network has the disadvantages of slow convergence rate and easy to fall into local minimum for network traffic *** performance of FNN is closely related to the number of nodes in the hidden layer,which affects the computational complexity,convergence rate and convergence *** proposed method uses PLS pruning algorithm to simply the network topology structure,which can obtain the ideal number of hidden layer nodes of the FNN,and then the prediction accuracy of network traffic is *** computer simulation results show that the proposed method has faster convergence rate and higher convergence accuracy compared with traditional FNN.
As an extensively used model for time series prediction, the Long-Short Term Memory (LSTM) neural network suffers from shortcomings such as high computational cost and large memory requirement, due to its complex stru...
详细信息
As an extensively used model for time series prediction, the Long-Short Term Memory (LSTM) neural network suffers from shortcomings such as high computational cost and large memory requirement, due to its complex structure. To address these problems, a PLS-based pruning algorithm is hereby proposed for a simplified LSTM (PSLSTM). First, a hybrid strategy is designed to simplify the internal structure of LSTM, which combines the structure simplification and parameter reduction for gates. Second, partial least squares (PLS) regression coefficients are used as the metric to evaluate the importance of the memory blocks, and the redundant hidden layer size is pruned by merging unimportant blocks with their most correlated ones. The Backpropagation Through Time (BPTT) algorithm is utilized as the learning algorithm to update the network parameters. Finally, several benchmark and practical datasets for time series prediction are used to evaluate the performance of the proposed PSLSTM. The experimental results demonstrate that the PLS-based pruning algorithm can achieve the trade-off between a good generalization ability and a compact network structure. The computational complexity is improved by the simple internal structure as well as the compact hidden layer size, without sacrificing prediction accuracy. (C) 2022 Elsevier B.V. All rights reserved.
暂无评论