Sample efficient deep reinforcement learning for Diwata microsatellite reaction wheel attitude control

作     者：Sarmiento, James-Andrew R. Tan, Vanessa H. Talampas, Marc Caesar R. Naval, Prospero C. 

作者机构：Department of Computer Science University of the Philippines Quezon City Philippines Electrical and Electronics Engineering Institute University of the Philippines Quezon City Philippines Space Technology Missions and Systems Bureau Philippine Space Agency Quezon City Philippines 

出 版 物：《Aerospace Systems》 (Aerosp. Syst.)

年 卷 期：2023年第6卷第1期

页      面：61-69页

核心收录：

主　　题：Reinforcement learning 

摘      要：The Philippines has launched Diwata satellites to undertake different scientific missions. Low-orbit microsatellites are prone to external disturbances affecting their pointing accuracy;hence, an autonomous attitude control mechanism is vital to its operations. Deep reinforcement learning (DRL) has been proven effective in learning optimal control. There has been prior work regarding using DRL for the satellite’s reaction wheel attitude control in Mission, Attitude, and Telemetry Analysis (MATA)—a simulation environment using Unity for Diwata satellites. However, results show that the applied methods are sample inefficient and still underperform on specific metrics against Diwata’s current attitude control system. In addition, using Unity’s Machine Learning Agents toolkit (ML-Agents) limits the training to Soft-Actor Critic (SAC) and Proximal Policy Optimization (PPO). This study aims to extend the prior research using Twin-Delayed Deep Deterministic Policy Gradient (TD3) and Prioritized Experience Replay (PER) to improve the performance and sample efficiency of the satellite agent. The training was done using OpenAI Gym connected to the MATA simulation environment. We conclude that TD3-PER outperforms the algorithms of SAC, PPO, and PID of the prior study in both sample efficiency and control performance. © 2022, Shanghai Jiao Tong University.