Optimizing Neural Network Training: A Markov Chain Approach for Resource Conservation

作     者：Wang, Ke Huang, Xianting Tan, Cong Yiu, Siu-Ming Chen, Zicong Lei, Xiaolin 

作者机构：College of Information Science and Technology Jinan University Guangzhou China Department of Computer Science the University of Hong Kong Hong Kong China 

出 版 物：《IEEE Transactions on Artificial Intelligence》 (IEEE. Trans. Artif. Intell.)

年 卷 期：2024年

页      面：1-14页

核心收录：

主　　题：Forecasting 

摘      要：The additional resource consumption generated during the repeated training processes of neural networks, including time and computational power costs, is a problem of significant concern. We, therefore, propose a method that utilizes Markov Chain to predict the training outcomes of neural networks with the same structure. This method’s training is based on prior experience to optimize the parameter adjustment process, thereby reducing the number of times training must be started from scratch and lowering time costs. By predicting training outcomes and reducing forward and backward propagation computations, among other factors, computational resource consumption significantly decreases. Simultaneously, since Markov Chain represents a clear mathematical model, the properties of probability transition offer greater interpretability compared to traditional methods. In an era where explainable artificial intelligence is equally crucial, a more transparent training method could have greater application potential in many important scenarios. The dual benefits they provide exemplify the advantage of our approach. Regarding the critical part, we have theoretically and experimentally demonstrated that, under certain conditions, the neural network training process possesses Markov property and becomes a Markov process after clustering. IEEE