Digital-Twin-Enhanced Deep Reinforcement Learning for Intelligent Omni-Surface Configurations in MU-MIMO Systems

作     者：Ye, Xiaowen Yu, Xianghao Fu, Liqun 

作者机构：Xiamen University School of Informatics Department of Communication Engineering Xiamen361005 China City University of Hong Kong Department of Electrical Engineering Hong Kong 

出 版 物：《IEEE Internet of Things Journal》 (IEEE Internet Things J.)

年 卷 期：2025年第12卷第9期

页      面：13005-13020页

核心收录：

学科分类：0711[理学-系统科学] 0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China Open Research Project Programme of the State Key Laboratory of Internet of Things for Smart City, University of Macau National Social Science Fund of China 

主　　题：Deep reinforcement learning 

摘      要：Intelligent omni-surface (IOS) is a promising technique to enhance the capacity of wireless networks, by reflecting and refracting the incident signal simultaneously. Traditional IOS configuration schemes, relying on all subchannels’ channel state information and user equipments’ mobility, are difficult to implement in complex realistic systems. Existing works attempt to address this issue employing deep reinforcement learning (DRL), but this method requires a lot of trial-and-error interactions with the external environment for efficient results and thus cannot satisfy the real-time decision making. To enable model-free and real-time IOS control, this article puts forth a new framework that integrates DRL and digital twins. As a first step, deep reinforcement learning IOS (DeepIOS), a DRL based IOS configuration scheme with the goal of maximizing the sum data rate, is developed to jointly optimize the phase-shift and amplitude of IOS in multiuser multiple-input-multiple-output (MU-MIMO) systems. Thereafter, in order to further reduce the computational complexity, DeepIOS introduces an action branch architecture, which decides two optimization variables in parallel in a separate fashion. Finally, a digital twin module is constructed through supervised learning as a preverification platform for DeepIOS, such that the decision making’s real-time can be guaranteed. The formulated framework is a closed-loop system, in which the physical space provides data to establish and calibrate the digital space, while the digital space generates a large number of experience samples for DeepIOS training and sends the trained parameters to the IOS controller for configurations. Numerical results show that compared with random and MAB schemes, the proposed framework attains a higher data rate and is more robust to different settings. Furthermore, the action branch architecture reduces DeepIOS’s computational complexity, and the digital twin module improves DeepIOS’s convergence speed