Reinforcement Learning-Based Control of DC-DC Buck Converter Considering Controller Time Delay

作     者：Lee, Donghun Kim, Bongseok Kwon, Soonhyung Nguyen, Ngoc-Duc Kyu Sim, Min Lee, Young Il 

作者机构：Seoul Natl Univ Sci & Technol Dept Elect & Informat Engn Seoul 01811 South Korea Seoul Natl Univ Sci & Technol Dept Data Sci Seoul 01811 South Korea 

出 版 物：《IEEE ACCESS》 (IEEE Access)

年 卷 期：2024年第12卷

页      面：118442-118452页

核心收录：

基　　金：National Research Foundation of Korea (NRF) - Ministry of Education [NRF-2019R1A6A1A03032119  NRF-2021R1A6A1A03039981] 

主　　题：Delay effects Buck converters Real-time systems Delays Pulse width modulation Inductors Tuning DC-DC power converters Reinforcement learning Digital signal processing DC-DC synchronous buck converter real-time deep reinforcement learning (RTDRL) digital signal processor (DSP) optimal control 

摘      要：Non-linearities and unmodeled dynamics in the control system inevitably degrade the quality and reliability of voltage stabilization performance in DC-DC buck converters. Reinforcement Learning (RL) is an emerging method to mitigate this issue. However, traditional RL typically necessitates significant computational resources and specialized processing units, thus being an economically unreasonable option. This paper proposes a high-performance RL-based method even suitable for a cost-effective Digital Signal Processor (DSP). To address the significant challenge of time delay in a DSP when training the RL agent, this paper adopts a Real-Time Deep Reinforcement Learning (RTDRL) approach that creates an augmented virtual decision process to eliminate the delay effect. The performance is validated through software simulation (PLECS) and an actual system, through which the proposed approach demonstrated superior performance compared to existing benchmarks, including existing approaches and artificial intelligence.