The accurate dynamic model of the chemical process is an important condition for the successful implementation of advanced control in the plant. In this paper, an efficient secondorderalgorithm for long short-term m...
详细信息
The accurate dynamic model of the chemical process is an important condition for the successful implementation of advanced control in the plant. In this paper, an efficient secondorderalgorithm for long short-term memory (LSTM) network training is proposed for chemical process intelligent identification. A novel Hessian inverse recursion method is adopted to achieve fast convergence and avoid the high-cost operation of the classic secondorder optimization method. Besides, more information is back propagated since the proposed method retains the real curvature information of the neural network. Considering the large amount of chemical process data, a sub-sampled recursive secondorder-stochastic gradient descent (SRSO-SGD) algorithm which uses sub-sampling method and hybrid strategy is proposed. The identification experiment on a delayed coker fractionator shows that the proposed sub-sampled neural network secondorder training algorithm has better performance than other learningalgorithms in terms of model identification accuracy and convergence speed. By adopting a hybrid strategy that performing Hessian inverse estimation every 3 training epochs, the expensive Hessian inverse calculation cost in the identification process is further reduced while low training and testing errors are maintained.
In this paper a new secondorder recursive learningalgorithm to multilayer feedforward network is proposed. This algorithm makes not only each layer errors of network but also secondorder derivative information fact...
详细信息
ISBN:
(纸本)7312012035
In this paper a new secondorder recursive learningalgorithm to multilayer feedforward network is proposed. This algorithm makes not only each layer errors of network but also secondorder derivative information factors backpropagate. And it is proved that it is equivalent to Newton iterative algorithm and has secondorder convergent speed. New algorithm achieves the recurrence calculation of Newton search directions and the inverse of Hessian matrices. Its calculation quantity is correspond to that of common recursive least squares algorithm. It is stated clearly that this new algorithm is superior to Karayiannis' secondorderalgorithm according to analysis of their properties.
暂无评论