Heterogeneous hypergraph transformer network with cross-modal future interaction for multi-agent trajectory prediction

作     者：Zhou, Xiangzheng Chen, Xiaobo Yang, Jian 

作者机构：Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Nanjing 210094 Peoples R China Nanjing Univ Sci & Technol Key Lab Intelligent Percept & Syst High Dimens Inf Minist Educ Nanjing 210094 Peoples R China Shandong Technol & Business Univ Sch Comp Sci & Technol Yantai 264005 Shandong Peoples R China 

出 版 物：《ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE》 (Eng Appl Artif Intell)

年 卷 期：2025年第144卷

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China [62376139, 61773184, U24A20330, 62361166670] Taishan Scholars Program of Shandong Province [tsqn202103118] Shandong Provincial Natural Science Foundation [ZR2023MF014] Yantai Science and Technology Development Plan [2022XDRH014] 

主　　题：Heterogeneous hypergraph transformer Future cross-modal interaction Multi-agent trajectory prediction Multimodal trajectory prediction 

摘      要：Predicting the future trajectories of traffic agents is crucial for ensuring the safety and efficiency of self-driving vehicles and navigation robots in complex environments. Accurate trajectory prediction enhances decisionmaking and collision avoidance, which are fundamental to the deployment of autonomous systems in realworld scenarios such as urban traffic, delivery logistics, and pedestrian navigation. However, this task remains challenging due to agents diverse interactions and personalized behavior based on their categories. Moreover, most trajectory prediction models neglect the interaction between generated multimodal trajectories, limiting their practical applicability. This paper proposes a heterogeneous hypergraph transformer network with cross-modal future interaction (HHT-CFI) for multi-agent trajectory prediction to address these challenges. Specifically, the trajectory coordinates are first fed into a temporal model using transformer encoders to capture temporal features. Next, a heterogeneous hypergraph transformer (HHT) is designed to model social influence between agents of different categories, capturing historical interaction features. A cross-modal future interaction (CFI) module is then developed to enable agents to perceive multiple possible future paths of their neighbors. Finally, multimodal trajectories are generated twice, where the second predictions refine the first by utilizing cross-modal future features. Extensive experiments on four datasets demonstrate the superior performance of our model over state-of-the-art methods, showcasing its potential for real-world applications in autonomous navigation and traffic management. The code of the proposed method will be publicly available at https://github. com/zhou00NJUST/HHT-CFI.