Deep Reinforcement Learning for Optimizing Multi-Hop Distributed Collaborative Task Offloading in R2X

作     者：Zhao, Wei Weng, Tangjie Cheng, Yu Liu, Zhi Kato, Nei 

作者机构：Anhui University of Technology School of Computer Science and Technology Anhui 243032 China University of Electro-Communications Department of Computer and Network Engineering Tokyo182-8585 Japan Tohoku University Graduate School of Information Sciences Sendai980-8577 Japan 

出 版 物：《IEEE Transactions on Vehicular Technology》 (IEEE Trans. Veh. Technol.)

年 卷 期：2025年第74卷第6期

页      面：9533-9548页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：Deep reinforcement learning 

摘      要：Roadside units (RSUs) with strong sensing abilities enhance the viability of the RSU-to-Everything (R2X) paradigm, offering crucial infrastructure support for Mobile Edge Computing (MEC) that enables real-time data processing and reduced delay. Since RSUs collect a large volume of data but have limited computing capability, data analysis tasks are usually offloaded to other network nodes, such as the cloud, other RSUs, or even vehicles. The multi-hop distributed collaborative task offloading scheme is expected to achieve high resource utilization efficiency and low task delay in this scenario, despite increasing energy consumption in data transmission. However, the highly dynamic nature of the R2X network topology makes it challenging for a node to independently select the next hop and collaboratively allocate tasks to neighbors in a multi-hop transmission path. Specifically, offloading decisions made by an individual node are influenced not only by its immediate neighbors but also by other nodes along the multi-hop path, referred to in this paper as the effect value. Additionally, the heterogeneity in computing resources and link delays among network nodes further increases the difficulty. To address these challenges, we first apply a Long Short-Term Memory (LSTM) model to predict and update the neighbors for each node while considering effect values, allowing them to independently adapt to environmental changes. Then, we design a two-layer Deep Reinforcement Learning (DRL) algorithm for network nodes to make decisions. The first-layer DRL algorithm is implemented by RSUs to determine task offloading modes. When an RSU decides to offload tasks to multiple vehicles for collaborative computing, the second-layer DRL algorithm is used by a vehicle to select its next hop vehicle and allocate tasks. Simulation results show that our proposed approach effectively adapts to topology changes in complex and highly dynamic network environments. Compared with existing methods,