Reinforcement-Learning-Based APT Defense for Large-Scale Smart Grids

作     者：Xiao, Liang Liu, Huanhuan Lv, Zefang Chen, Yifan Lin, Zhiping Du, Yousong 

作者机构：Xiamen Univ Dept Informat & Commun Engn Xiamen 361005 Peoples R China Fudan Univ Sch Comp Sci Shanghai 200433 Peoples R China 

出 版 物：《IEEE INTERNET OF THINGS JOURNAL》 (IEEE Internet Things J.)

年 卷 期：2025年第12卷第9期

页      面：11917-11925页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China [U21A20444] National Key Research and Development Program of China [2023YFB3107603] 

主　　题：Maintenance engineering Smart grids Accuracy Data protection Security Meters Data models Costs Resource management Q-learning Advanced persistent threat (APTs) large-scale smart grids meter data management systems (MDMSs) reinforcement learning (RL) 

摘      要：Reinforcement-learning-(RL)-based advanced persistent threats (APTs) defense schemes choose the scan interval to enhance the detection accuracy, and the data protection level can be further improved by optimizing the repair rate corresponding to methods, such as malicious files removal, security software version update and passwords reset, each requiring different computational resources, such as CPUs. In this article, we propose an RL-based APT defense scheme to optimize both the continuous scan interval of metering data and repair rate to mitigate the potential loss for meter data management systems (MDMSs) in smart grids with large-scale meters. Based on the size of the metering data stored at MDMS and the compromised data, the data tag granularity and the number of CPUs for repairing, neural networks extract the state feature, address the quantization error of defense policy, and update the weights based on the APT defense experiences and the shared weights of neighboring MDMSs to improve the defense performance as a weighted sum of the defense duration, detection accuracy and data protection level. The computational complexity of the proposed scheme and the performance bounds according to the Nash Equilibrium of the game between the MDMS and the APT attacker are provided. Simulation results based on three MDMSs that receive the metering data from 50-150 smart meters and an APT attacker with the selected attack interval up to 5 s show the performance gain over the benchmark based on the optimal control theory and Q-learning in large-scale smart grids.