The performance of models based on a Feedforward Neural Network depends strongly on the initial estimate of weights and the number of units in the hidden layers. This work presents a new method for the initialization ...
详细信息
The performance of models based on a Feedforward Neural Network depends strongly on the initial estimate of weights and the number of units in the hidden layers. This work presents a new method for the initialization of weights and definition of the number of hidden units in the identification of Multiple Input Single Output models associated with regression problems. The initialization strategy consists of a complete linearization of the network with only one neuron unit around an equilibrium point and the determination of the initial weights through the maximum approximation of the linearized model to the Optimal Linear Regressor whose solution can be obtained analytically. The constructive algorithm performs a gradual increase in the number of hidden units in such a way that at each training only weights associated with new hidden units are randomly initialized, while weights obtained from previous training are used as initial guess for the subsequent ones. The proposed method was compared to the classical random initialization method and to the Extreme Learning Machine (a typical gradient-free learning algorithm), both of which also involve a constructive approach to define the final network. The methods were applied to 11 real datasets widely used as benchmarks for regression problems. The proposed method performed better than the other approaches in 8-9 case studies and was able to ensure a monotonic decrease in the loss function with an increasing number of hidden units.
constructive algorithm provides a gradually building mechanism by increasing nodes from zero. By this means, the neural network can independently and efficiently determine its structure. However, this mechanism has an...
详细信息
constructive algorithm provides a gradually building mechanism by increasing nodes from zero. By this means, the neural network can independently and efficiently determine its structure. However, this mechanism has an essential issue: the algorithm that adds nodes one by one is too greedy to keep an efficient construction way and the global optimal solution may be missed. Therefore, this paper proposes a novel grafting mechanism to add block nodes of any number by training a sub-network during the construction. Then, a fast-training approach of the added block neurons is presented by selecting a small sub-network from the large initialized network and the corresponding grafting constructive algorithm (GCA) is established. To obtain a compact network structure, a fine-tuning scheme is developed according to GCA to adjust all parameters as a hybrid fashion and the hidden weights are extended to deal with matrix input in image classification. The experimental results on regression and classification tasks demonstrate that the proposed GCA can achieve a more compact network than other constructive algorithms and a faster error convergence rate than traditional gradient-based optimization algorithms.
This work presents the development and validation of two virtual analyzers (density and Melt Index (MI)) for quality monitoring and control of the final product in an industrial unit of Linear Polyethylene (LPE). Both...
详细信息
This work presents the development and validation of two virtual analyzers (density and Melt Index (MI)) for quality monitoring and control of the final product in an industrial unit of Linear Polyethylene (LPE). Both models are based on Feedforward Neural Networks which are improved through a strategy involving the initial estimation of weights and a constructive algorithm to define the number of hidden units. The initialization strategy is based on linearization of the neural model with only one hidden unit (nonlinear model) and subsequent optimization of this model by maximizing its similarity to the standard linear regression model whose solution is obtained analytically. The Initial Neural Model (INM) is then used as a starting point for a gradual increase in the number of hidden units. In a validation test involving MI and density values collected over 2 years of operation, the neural model is able to predict these properties with mean percentage errors equal to 0.81% (MI) and 0.04% (density) and determination coefficients equal to 0.970 (MI) and 0.983 (density). The population coefficient estimated in all tests involving grade transitions (0.96) shows a strong linear correlation between the proposed model and laboratory measurements.
This paper presents a new method for initializing weights in a Feedforward Neural Network (FNN) with a single hidden layer combined with a constructive approach to define the number of hidden units associated with the...
详细信息
This paper presents a new method for initializing weights in a Feedforward Neural Network (FNN) with a single hidden layer combined with a constructive approach to define the number of hidden units associated with the best classification performance. The strategy consists of defining an initial number of hidden units according to the classification problem, the linearization of the whole network around an equilibrium point and the determination of the initial weights and bias through the maximum approximation of the linearized model to the Optimal Linear Classifier (OLC) whose solution can be obtained analytically. The constructive algorithm comprises a gradual increase in the number of hidden units in such a way that at each training only the weights and bias associated with the new hidden units are initiated randomly while the weights and bias obtained from previous training are used as initial guesses. Additionally, the constructive algorithm seeks to ensure that the loss function of the trained networks decreases with the successive additions of hidden units. The proposed approach (Weight Initialization based on the Linearization of the Whole Neural Network combined with a new constructive algorithm, WILWNN-CA) is applied to synthetic and real datasets widely used as benchmark for multi-class classification problems. The comparison with conventional random weight initialization and other approaches involving different network topologies (and initialization strategies) shows that the proposed method is efficient and capable of providing success rates (correct classification rates) higher or similar to those achieved with existing methods.
In this paper, a novel constructive algorithm, named fast cascade neural network (FCNN), is proposed to design the fully connected cascade feedforward neural network (FCCFNN). First, a modified index, based on the ort...
详细信息
In this paper, a novel constructive algorithm, named fast cascade neural network (FCNN), is proposed to design the fully connected cascade feedforward neural network (FCCFNN). First, a modified index, based on the orthogonal least square method, is derived to select new hidden units from candidate pools. Each hidden unit leads to the maximal reduction of the sum of squared errors. Secondly, the input weights and biases of hidden units are randomly generated and remain unchanged during the learning process. The weights, which connect the input and hidden units with the output units, are calculated after all necessary units have been added. Thirdly, the convergence of FCNN is guaranteed in theory. Finally, the performance of FCNN is evaluated on some artificial and real-world benchmark problems. Simulation results show that the proposed FCNN algorithm has better generalization performance and faster learning speed than some existing algorithms. (C) 2015 Elsevier B.V. All rights reserved.
Theories inspired by the working and the structure of the human brain have been applied in various problems in computer vision, such as Artificial Neural Networks (ANNs). The concept of receptive and inhibitory fields...
详细信息
ISBN:
(纸本)9781509061839
Theories inspired by the working and the structure of the human brain have been applied in various problems in computer vision, such as Artificial Neural Networks (ANNs). The concept of receptive and inhibitory fields have been adopted with success to improve the capability of the ANNs and are present in the deep learning models, such as ConvNet and LIPNet, that have been successfully used in many computer vision tasks. However, both shallow and deep ANN models need some expertise to define their architecture, as well as the definition of some of their parameters. A recently introduced ANN, called CANet, embeds the concepts of receptive fields, lateral inhibition, and autoassociative memory using a constructive algorithm that requires few parameters to perform its learning process. This paper presents a new constructive-pruning algorithm for CANet that contains even fewer parameters and can self-choose the quantity of neurons in the constructive layer of the model, called CANet-2. Also, we analyze the behavior of the model with the use of activation functions from the ReLU family in its constructive layer. Experiments in facial expression recognition showed that the proposed constructive algorithm along with the SoftPlus activation function improved CANet-2 in relation to the original version.
Given n subspaces of a finite-dimensional vector space over a fixed finite field F, we wish to find a linear layout V_1, V_2, ..., V_n of the subspaces such that dim((V_1+V_2+... + V_i)n(V_(i+1)+... +V_n)) ≤ k for al...
详细信息
ISBN:
(纸本)9781510819672
Given n subspaces of a finite-dimensional vector space over a fixed finite field F, we wish to find a linear layout V_1, V_2, ..., V_n of the subspaces such that dim((V_1+V_2+... + V_i)n(V_(i+1)+... +V_n)) ≤ k for all i; such a linear layout is said to have width at most k. When restricted to 1-dimensional subspaces, this problem is equivalent to computing the path-width of an F-represented matroid in matroid theory and computing the trellis-width (or minimum trellis statecomplexity) of a linear code in coding theory. We present a fixed-parameter tractable algorithm to construct a linear layout of width at most k, if it exists, for input subspaces of a finite-dimensional vector space over F. As corollaries, we obtain a fixed-parameter tractable algorithm to produce a path-decomposition of width at most k for an input F-represented matroid of path-width at most k, and a fixed-parameter tractable algorithm to find a linear rank-decomposition of width at most k for an input graph of linear rank-width at most k. In both corollaries, no such algorithms were known previously. Our approach is based on dynamic programming combined with the idea developed by Bodlaender and Kloks (1996) for their work on path-width and tree-width of graphs. It was previously known that a fixed-parameter tractable algorithm exists for the decision version of the problem for matroid path-width; a theorem by Geelen, Gerards, and Whittle (2002) implies that for each fixed finite field F, there are finitely many forbidden F-representable minors for the class of matroids of path-width at most k. An algorithm by Hlineny (2006) can detect a minor in an input F-represented matroid of bounded branch-width. However, this indirect approach would not produce an actual pathdecomposition even if the complete list of forbidden minors were known. Our algorithm is the first one to construct such a path-decomposition and does not depend on the finiteness of forbidden minors.
We propose a new constructive algorithm, called HAPE3 D, which is a heuristic algorithm based on the principle of minimum total potential energy for the 3D irregular packing problem, involving packing a set of irregul...
详细信息
We propose a new constructive algorithm, called HAPE3 D, which is a heuristic algorithm based on the principle of minimum total potential energy for the 3D irregular packing problem, involving packing a set of irregularly shaped polyhedrons into a box-shaped container with fixed width and length but unconstrained height. The objective is to allocate all the polyhedrons in the container, and thus minimize the waste or maximize profit. HAPE3 D can deal with arbitrarily shaped polyhedrons, which can be rotated around each coordinate axis at different angles. The most outstanding merit is that HAPE3 D does not need to calculate no-fit polyhedron(NFP), which is a huge obstacle for the 3D packing problem. HAPE3 D can also be hybridized with a meta-heuristic algorithm such as simulated annealing. Two groups of computational experiments demonstrate the good performance of HAPE3 D and prove that it can be hybridized quite well with a meta-heuristic algorithm to further improve the packing quality.
Inspired on decision trees and evolutionary algorithms, this paper proposes a learning algorithm of constructive neural networks that relies on three principles: to layout the neurons in a tree-like structure;to train...
详细信息
ISBN:
(纸本)9781509006212
Inspired on decision trees and evolutionary algorithms, this paper proposes a learning algorithm of constructive neural networks that relies on three principles: to layout the neurons in a tree-like structure;to train each neuron individually;and, to optimize all the weights using an evolutionary approach. This way, it is expected to advance in two main questions concerning multilayer perceptrons (MLPs): how to determine the network architecture and how to build models that are more comprehensible. Based on the normalized information gain of each attribute, the algorithm builds the network architecture. In the process, it automatically creates a set of training examples for each individual neuron and executes single-cell learning. Once the network is created and trained, particle swarm optimization is utilized to evolve the connections of the network. Five metrics were utilized to validate the method when compared to decision trees and MLPs: accuracy, sensitivity, specificity, precision and comprehensibility. The experiments were executed in thirteen different databases and the results suggest that the proposed algorithm can generate neural networks with good classification performance and more comprehensible.
Yard Allocation Problem (YAP) is a frequently problem observed in port logistic operations, and consists in a temporary assignment of cargo containers, both export and import, to blocks storage located in the operatio...
详细信息
ISBN:
(纸本)9781467387569
Yard Allocation Problem (YAP) is a frequently problem observed in port logistic operations, and consists in a temporary assignment of cargo containers, both export and import, to blocks storage located in the operation zone of seaports. The principal objective of YAP resolution is to improve the operations efficiency both terminal port as shipping companies that use the seaport, which allow to decrease operational cost and increasing the global efficiency. In this sense, this work proposes to solve the YAP using a constructive algorithm based in restrictions for each container. Official data of Arica Seaport - Chile were used for defining an experimental scenario, which allowed obtaining promising results. The constructive algorithm represents a novel approach because allows getting solutions in a simple and fast way, while satisfying the existing set of constrains associated with the general operation model.
暂无评论