UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience Replay

作     者：Luo, Xuqiong Wang, Qiyuan Gong, Hongfang Tang, Chao 

作者机构：Changsha Univ Sci & Technol Sch Math & Stat Changsha 410114 Peoples R China Changsha Univ Sci & Technol Sch Comp & Commun Engn Changsha Peoples R China 

出 版 物：《IEEE ACCESS》 (IEEE Access)

年 卷 期：2024年第12卷

页      面：38017-38029页

核心收录：

基　　金：Excellent Youth Project of Education Department of Hunan Province 

主　　题：Autonomous aerial vehicles Path planning Heuristic algorithms Training Approximation algorithms Machine learning algorithms Stability analysis Deep reinforcement learning Vehicle dynamics Performance evaluation UAV path planning deep reinforcement learning prioritized experience replay average TD3 algorithm 

摘      要：Path planning is one of the important components of the Unmanned Aerial Vehicle (UAV)mission, and it is also the key guarantee for the successful completion of the UAV s mission. The traditionalpath planning algorithm has certain limitations and deficiencies in the complex dynamic *** at the dynamic complex obstacle environment, this paper proposes an improved TD3 algorithm,which enables the UAV to complete the autonomous path planning through online learning and continuoustrial and error. The algorithm changes the experience pool of TD3 algorithm to priority experience replay,so that the agent can distinguish the importance of empirical samples, improve the sampling efficiency ofthe algorithm, and reduce the training time. The average TD3 is proposed, and the average value ofQ1Q2is taken when the target value is updated to solve the problem of overestimating theQvalue while avoidingunderestimating theQvalue, so that the improved algorithm has better stability and can adapt to variouscomplex obstacle environments. A new reward function is set up, so that each step of the UAV action canreceive reward feedback, which solves the problem of sparse reward in deep reinforcement learning. Theexperimental results show that this method can train the UAV to reach the target safely and quickly in amulti-obstacle environment. Compared with DDPG, SAC and traditional TD3, the path planning successrate of this algorithm is higher than that of the other three algorithms, and the collision rate is lower than thatof the comparison algorithm, which has better path planning performance