A Reinforcement Learning Approach to Autonomous Decision Making of Intelligent Vehicles on Highways

作     者：Xu, Xin Zuo, Lei Li, Xin Qian, Lilin Ren, Junkai Sun, Zhenping 

作者机构：Natl Univ Def Technol Inst Unmanned Syst Coll Intelligence Sci & Technol Changsha 410073 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS》 (IEEE Trans. Syst. Man Cybern. Syst.)

年 卷 期：2020年第50卷第10期

页      面：3884-3897页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China [61751311  U1564214  61611540348] 

主　　题：Autonomous decision-making intelligent driving vehicles Markov decision processes (MDPs) multiobjective,reinforcement learning (RL) value function approximation 

摘      要：Autonomous decision making is a critical and difficult task for intelligent vehicles in dynamic transportation environments. In this paper, a reinforcement learning approach with value function approximation and feature learning is proposed for autonomous decision making of intelligent vehicles on highways. In the proposed approach, the sequential decision making problem for lane changing and overtaking is modeled as a Markov decision process with multiple goals, including safety, speediness, smoothness, etc. In order to learn optimized policies for autonomous decision-making, a multiobjective approximate policy iteration (MO-API) algorithm is presented. The features for value function approximation are learned in a data-driven way, where sparse kernel-based features or manifold-based features can be constructed based on data samples. Compared with previous RL algorithms such as multiobjective Q-learning, the MO-API approach uses data-driven feature representation for value and policy approximation so that better learning efficiency can be achieved. A highway simulation environment using a 14 degree-of-freedom vehicle dynamics model was established to generate training data and test the performance of different decision-making methods for intelligent vehicles on highways. The results illustrate the advantages of the proposed MO-API method under different traffic conditions. Furthermore, we also tested the learned decision policy on a real autonomous vehicle to implement overtaking decision and control under normal traffic on highways. The experimental results also demonstrate the effectiveness of the proposed method.