Symbolic Visual Reinforcement Learning: A Scalable Framework With Object-Level Abstraction and Differentiable Expression Search

作     者：Zheng, Wenqing Sharan, S. P. Fan, Zhiwen Wang, Kevin Xi, Yihan Wang, Zhangyang 

作者机构：Univ Texas Austin Dept Elect Engn Austin TX 78712 USA 

出 版 物：《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 (IEEE Trans Pattern Anal Mach Intell)

年 卷 期：2025年第47卷第1期

页      面：400-412页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China Foundation of Key Laboratory of Artificial Intelligence Ministry of Education, P.R. China 

主　　题：Visualization Optimization Reinforcement learning Representation learning Neural networks Vegetation Planning Object detection symbolic expression symbolic regression visual reinforcement learning (RL) 

摘      要：Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are difficult to interpret and deploy efficiently. More recent SRL frameworks have shown that high-level domain-specific programming logic can be designed to handle both policy learning and symbolic planning. However, these approaches rely on coded primitives with little feature learning, and when applied to high-dimensional visual scenes, they can suffer from scalability issues and perform poorly when images have complex object interactions. To address these challenges, we propose Differentiable Symbolic Expression Search (DiffSES), a novel symbolic learning approach that discovers discrete symbolic policies using partially differentiable optimization. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions, while also incorporating the strengths of neural networks for feature learning and optimization. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more and scalable than state-of-the-art SRL methods, with a reduced amount of symbolic prior knowledge.