DynamicVLN: Incorporating Dynamics into Vision-and-Language Navigation Scenarios

作     者：Sun, Yanjun Qiu, Yue Aoki, Yoshimitsu 

作者机构：Keio Univ Dept Elect & Elect Engn Fac Sci & Technol 3-14-1 HiyoshiKohoku Ku Yokohama 2238522 Japan Natl Inst Adv Ind Sci & Technol 1-1-1 Umezono Tsukuba 3058560 Japan 

出 版 物：《SENSORS》 (Sensors)

年 卷 期：2025年第25卷第2期

页      面：364-364页

核心收录：

学科分类：0710[理学-生物学] 071010[理学-生物化学与分子生物学] 0808[工学-电气工程] 07[理学] 0804[工学-仪器科学与技术] 0703[理学-化学] 

基　　金：JST SPRING JPMJSP2123 

主　　题：vision-and-language navigation dynamic change decision-making 

摘      要：Traditional Vision-and-Language Navigation (VLN) tasks require an agent to navigate static environments using natural language instructions. However, real-world road conditions such as vehicle movements, traffic signal fluctuations, pedestrian activity, and weather variations are dynamic and continually changing. These factors significantly impact an agent s decision-making ability, underscoring the limitations of current VLN models, which do not accurately reflect the complexities of real-world navigation. To bridge this gap, we propose a novel task called Dynamic Vision-and-Language Navigation (DynamicVLN), incorporating various dynamic scenarios to enhance the agent s decision-making abilities and adaptability. By redefining the VLN task, we emphasize that a robust and generalizable agent should not rely solely on predefined instructions but must also demonstrate reasoning skills and adaptability to unforeseen events. Specifically, we have designed ten scenarios that simulate the challenges of dynamic navigation and developed a dedicated dataset of 11,261 instances using the CARLA simulator (ver.0.9.13) and large language model to provide realistic training conditions. Additionally, we introduce a baseline model that integrates advanced perception and decision-making modules, enabling effective navigation and interpretation of the complexities of dynamic road conditions. This model showcases the ability to follow natural language instructions while dynamically adapting to environmental cues. Our approach establishes a benchmark for developing agents capable of functioning in real-world, dynamic environments and extending beyond the limitations of static VLN tasks to more practical and versatile applications.