Condition-based maintenance via Markov decision processes: A review

作     者：Xiujie ZHAO Piao CHEN Loon Ching TANG Xiujie ZHAO;Piao CHEN;Loon Ching TANG

作者机构：College of Management and EconomicsTianjin UniversityTianjin 300072China ZJU-UIUC InstituteZhejiang UniversityHaining 314400China Department of Industrial Systems Engineering and ManagementNational University of SingaporeSingapore 117576Singapore 

出 版 物：《Frontiers of Engineering Management》 (工程管理前沿(英文版))

年 卷 期：2025年第12卷第2期

页      面：330-342页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Natural Science Foundation of China(Grant Nos.72401253,72371182,72002149,and 72271154) and the National Social Science Fund of China(23CGL018) the State Key Laboratory of Biobased Transportation Fuel Technology,China(Grant No.512302-X02301) a start-up grant from the ZJU-UIUC Institute at Zhejiang University(Grant No.130200-171207711) 

主　　题：reliability degradation modeling dynamic programming reinforcement learning sequential decision problems 

摘      要：The optimization of condition-based maintenance (CBM) poses challenges due to the rapid advancement of monitoring technologies. Traditional CBM research has mainly relied on theory-driven approaches, which lead to the development of several effective maintenance models characterized by their wide applicability and attractiveness. However, when the system reliability model becomes complex, such methods may run into intractable cost models. The Markov decision process (MDP), a classic framework for sequential decision making, has drawn increasing attention for optimization of CBM optimization due to its appealing tractability and pragmatic applicability across different problems. This paper presents a review of research that optimizes CBM policies using MDP, with a focus on mathematical modeling and optimization methods to enable action. We have organized the review around several key components that are subject to similar mathematical modeling constraints, including system complexity, the availability of system conditions, and diverse criteria of decision-makers. An increase in interest has led to the optimization of CBM for systems possessing increasing numbers of components and sensors. Then, the review focuses on joint optimization problems with CBM. Finally, as an important extension to traditional MDPs, reinforcement learning (RL) based methods are also reviewed as ways to optimize CBM policies. This paper provides significant background research for researchers and practitioners working in reliability and maintenance management, and gives discussions on possible future research directions.