Trying to extract features from complex sequential data for classification and prediction problems is an extremely difficult task. This task is even more challenging when both the upstream and downstream information o...
详细信息
Trying to extract features from complex sequential data for classification and prediction problems is an extremely difficult task. This task is even more challenging when both the upstream and downstream information of a time-series is important to process the sequence at a specific time-step. One typical problem which falls in this category is Protein secondary Structure Prediction (PSSP). Recurrent Neural Networks (RNNs) have been successful in handling sequential data. These methods are demanding in terms of time and space efficiency. On the other hand, simple Feed-Forward Neural Networks (FFNNs) can be trained really fast with the Backpropagation algorithm, but in practice they give poor results in this category of problems. The Hessian Free Optimization (HFO) algorithm is one of the latest developments in the field of Artificial Neural Network (ANN) training algorithms which can converge faster and more accurately. In this paper, we present the implementation of simple FFNNs trained with the powerful HFO second-orderlearning algorithm for the PSSP problem. In our approach, a single FFNN trained with the HFO learning algorithm can achieve an approximately 79.6% per residue (Q(3)) accuracy on the PISCES dataset. Despite the simplicity of our method, the results are comparable to some of the state of the art methods which have been designed for this problem. A majority voting ensemble method and filtering with Support Vector Machines have also been applied, which increase our results to 80.4% per residue (Q(3)) accuracy. Finally, our method has been tested on the CASP13 independent dataset and achieved 78.14% per residue (Q(3)) accuracy. Moreover, the HFO does not require tuning of any parameters which makes training much faster than other state of the art methods, a very important feature with big datasets and facilitates fast training of FFNN ensembles.
The paper presents the efficient training program of multilayer feedforward neural networks. It is based on the best secondorder optimization algorithms including variable metric and conjugate gradient as well as app...
详细信息
The paper presents the efficient training program of multilayer feedforward neural networks. It is based on the best secondorder optimization algorithms including variable metric and conjugate gradient as well as application of directional minimization in each step. Its efficiency is proved on the standard rests, including parity, dichotomy, logistic and two-spiral problems. The application of the algorithm to the solution of higher dimensionality problems like deconvolution, separation of sources and identification of nonlinear dynamic plant are also given and discussed. It is shown that the appropriately trained neural network can be used for the nonconventional solution of these standard signal processing tasks with satisfactory accuracy. The results of numerical experiments are included and discussed in the paper. Copyright (C) 1996 Elsevier Science Ltd.
暂无评论