Sparse Learning-Based Approximate Dynamic Programming With Barrier Constraints

作     者：Greene, Max L. Deptula, Patryk Nivison, Scott Dixon, Warren E. 

作者机构：Univ Florida Dept Mech & Aerosp Engn Gainesville FL 32611 USA Charles Stark Draper Lab Inc Percept & Auton Grp Cambridge MA 02139 USA Air Force Res Lab Munit Directorate Eglin AFB FL 32542 USA 

出 版 物：《IEEE CONTROL SYSTEMS LETTERS》 (IEEE Control Syst. Lett.)

年 卷 期：2020年第4卷第3期

页      面：743-748页

核心收录：

基　　金：Office of Naval Research [N00014-13-1-0151] Naval Engineering Education Consortium [N00174-18-10003] Air Force Office of Scientific Research (AFOSR) [FA9550-18-1-0109, FA9550-19-1-0169] National Science Foundation Directorate For Engineering Div Of Civil, Mechanical, & Manufact Inn Funding Source: National Science Foundation 

主　　题：Extrapolation Safety Optimal control Function approximation Symmetric matrices Transforms Trajectory Data-based control dynamic programming nonlinear control optimal control reinforcement learning 

摘      要：This letter provides an approximate online adaptive solution to the infinite-horizon optimal control problem for control-affine continuous-time nonlinear systems while formalizing system safety using barrier certificates. The use of a barrier function transform provides safety certificates to formalize system behavior. Specifically, using a barrier function, the system is transformed to aid in developing a controller which maintains the system in a pre-defined constrained region. To aid in online learning of the value function, the state-space is segmented into a number of user-defined segments. Off-policy trajectories are selected in each segment, and sparse Bellman error extrapolation is performed within each respective segment to generate an optimal policy within each segment. A Lyapunov-like stability analysis is included which proves uniformly ultimately bounded regulation in the presence of the barrier function transform and discontinuities. Simulation results are provided for a two-state dynamical system to compare the performance of the developed method to existing methods.