Gait Learning Reproduction for Quadruped Robots Based on Experience Evolution Proximal Policy Optimization

作     者：Li, Chunyang Zhu, Xiaoqing Ruan, Xiaogang Liu, Xinyuan Zhang, Siyuan 

作者机构：Faculty of Information Technology Beijing University of TechnologyBeijing Key Laboratory of Computational Intelligence and Intelligent SystemEngineering Research Center of Digital Community of Ministry of Education Beijing100124 China 

出 版 物：《Journal of Shanghai Jiaotong University (Science)》 (J. Shanghai Jiaotong Univ. Sci.)

年 卷 期：2023年

页      面：1-9页

核心收录：

基　　金：Foundation item: the National Natural Science Foundation of China (No. 62103009) 

主　　题：Bionics 

摘      要：Bionic gait learning of quadruped robots based on reinforcement learning has become a hot research topic. The proximal policy optimization (PPO) algorithm has a low probability of learning a successful gait from scratch due to problems such as reward sparsity. To solve the problem, we propose a experience evolution proximal policy optimization (EEPPO) algorithm which integrates PPO with priori knowledge highlighting by evolutionary strategy. We use the successful trained samples as priori knowledge to guide the learning direction in order to increase the success probability of the learning algorithm. To verify the effectiveness of the proposed EEPPO algorithm, we have conducted simulation experiments of the quadruped robot gait learning task on Pybullet. Experimental results show that the central pattern generator based radial basis function (CPG-RBF) network and the policy network are simultaneously updated to achieve the quadruped robot’s bionic diagonal trot gait learning task using key information such as the robot’s speed, posture and joints information. Experimental comparison results with the traditional soft actor-critic (SAC) algorithm validate the superiority of the proposed EEPPO algorithm, which can learn a more stable diagonal trot gait in flat terrain. © 2023, Shanghai Jiao Tong University.