检索结果-内蒙古大学图书馆

multi-agent proximal policy optimization based efficient user association and resource allocation in UAV-assisted Heterogeneous Cellular Networks

引用

COMPUTER COMMUNICATIONS 2025年 238卷

作者： Song, Yueqian Zeng, Qingtian Chen, Geng Yuan, Guiyuan Duanc, Hua Shandong Univ Sci & Technol Coll Elect & Informat Engn Qingdao 266590 Peoples R China Shandong Univ Sci & Technol Coll Comp Sci & Engn Qingdao 266590 Peoples R China Shandong Univ Sci & Technol Coll Math & Syst Sci Qingdao 266590 Peoples R China

The rapid growth in wireless communication demands has placed unprecedented pressure on modern networks, particularly concerning capacity enhancement and coverage expansion. Heterogeneous Cellular Networks (HCNs), with their flexible multi-layered architecture, present a promising solution to these challenges. However, these networks face considerable complexity and resource constraints. Consequently, developing efficient User Association and Resource Allocation (UARA) schemes is essential for establishing sustainable, high-performance wireless communication systems. In this work, we establish a paradigm for an Unmanned Aerial Vehicle (UAV)-assisted HCN with multiple User Equipment (UEs). An optimization scheme for UAV deployment and UARA policy is proposed to maximize the long-term system utility while ensuring quality of service for UEs. Specifically, we employ an efficient greedy deployment algorithm to dynamically update UAV locations and maximize the coverage utility. For UARA decision-making, we introduce a dynamic non-convex mixed-integer nonlinear programming problem and model it as a partially observable Markov decision process. Subsequently, an Independent Reward-based Hybrid action space multi-agent proximal policy optimization algorithm (IRH-MAPPO) is proposed. Additionally, we utilize a centralized training and distributed execution framework and incorporate value normalization and action masking to improve its efficiency. Experimental results demonstrate that our algorithm significantly outperforms the baselines in terms of system utility, particularly in Spectral Efficiency (SE) and Energy Efficiency (EE), which confirms its effectiveness, superiority, and scalability.

关键词： Heterogeneous cellular networks Unmanned aerial vehicle User association Resource allocation multi-agent proximal policy optimization

来源：评论

学校读者我要写书评

暂无评论

Joint Trajectory and Resource optimization of MEC-Assisted UAVs in Sub-THz Networks: A Resources-Based multi-agent proximal policy optimization DRL With Attention Mechanism

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2024年第2期73卷 2003-2016页

作者： Park, Yu Min Hassan, Sheikh Salman Tun, Yan Kyaw Han, Zhu Hong, Choong Seon Kyung Hee Univ Dept Comp Sci & Engn Yongin 17104 South Korea KTH Royal Inst Technol Sch Elect Engn & Comp Sci Div Network & Syst Engn Teletraff Syst S-11428 Stockholm Sweden Univ Houston Elect & Comp Engn Dept Houston TX 77004 USA

The use of Terahertz (THz) technology in sixth-generation (6G) networks will bring high-speed and capacity data services. But limitations like molecular absorption, rain attenuation, and limited coverage range cause communication losses. To overcome these losses and improve coverage in rural areas, a high number of base stations are required. Hence, an aerial communication platform, which uses line-of-sight (LoS) communication to avoid losses, is needed. To address this, we study the deployment and optimization of multi-access edge computing (MEC)-powered unmanned aerial vehicle (UAV) for sub-THz communication in remote areas. To this end, we solve an optimization problem to minimize energy consumption and delay for MEC-UAV and mobile users. The formulated problem is a mixed-integer non-linear programming (MINLP) problem. As the problem is an MINLP, we decompose the main problem into two subproblems. Due to its convex nature, we solve the first subproblem with a standard optimization solver, i.e., CVXPY. To solve the second subproblem, we design a resources-based multi-agent proximal policy optimization (RMAPPO) deep reinforcement learning (DRL) algorithm with an attention mechanism. The considered attention mechanism is utilized for encoding a diverse number of observations. This is designed by the network coordinator to provide a differentiated fit reward to each agent in the network. The simulation results show that the proposed algorithm outperforms the benchmark and yields a network utility that is 2.22%, 15.55%, and 17.77% more than the benchmarks.

关键词： Unmanned aerial vehicles (UAVs) mobile-edge computing resource allocation sub-terahertz communication multi-agent proximal policy optimization attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Optimal scheduling of shared autonomous electric vehicles with multi-agent reinforcement learning: A MAPPO-based approach

引用

NEUROCOMPUTING 2025年 622卷

作者： Tian, Jingjing Jia, Hongfei Wang, Guanfeng Huang, Qiuyang Wu, Ruiyi Gao, Heyao Liu, Chao Jilin Univ Coll Transportat Changchun 130022 Peoples R China

The advent of shared autonomous electric vehicles (SAEVs) is expected to decarbonize the transport sector and enable large-scale mobility on demand. Considering the highly dynamic nature of operation environments, it is appealing to investigate the optimal schedule decisions, comprising order-matching, relocation, and charge/ discharge with the V2G technology. Recent literature has investigated these schedule decisions with deep reinforcement learning (DRL) independently, neglecting the collaborative interactions among multiple autonomous vehicles. To this end, the fully dynamic schedule task is integrally described as a partially observable Markov decision process (POMDP) under the constraints of time-dependent demand, spatial-temporal varying electricity price, available vehicles and chargers. Then, we design a robust and extensible framework that combines multi-agent proximal policy optimization (MAPPO) and a binary linear programming (BLP) model to solve the optimization problem, in which SAEVs are regarded as agents to make decisions according to their local observations. Additionally, observation normalization and action masking are utilized to improve training efficiency. Numerical experiments are conducted with New York City yellow taxi data to simulate the real-time SAEV operating environment. These results demonstrate that the proposed MAPPO approach is expected to enhance the operators' revenue while improving the customer service rate compared to benchmark algorithms.

关键词： Shared autonomous electric vehicles Optimal scheduling multi-agent proximal policy optimization Binary linear programming

来源：评论

学校读者我要写书评

暂无评论

multi-agent Deep Reinforcement Learning-Based Resource Allocation for Cognitive Radio Networks

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2025年第3期74卷 4744-4757页

作者： Mei, Ruru Wang, Zhugang Chinese Acad Sci Natl Space Sci Ctr Key Lab Elect & Informat Technol Space Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Elect Elect & Commun Engn Beijing 100049 Peoples R China Chinese Acad Sci Natl Space Sci Ctr Beijing 100190 Peoples R China

In this paper, we investigate the spectrum access and power control problem in a heterogeneous cellular and vehicular communication network comprising both cellular links and vehicle-to-vehicle (V2V) links. We formulate the spectrum access and power control problem as a fully cooperative multi-agent reinforcement learning task, where each V2V link receives an identical reward at each time step. This strategy caters to the heterogeneous Quality-of-Service requirements, including the high reliability and low latency requirements for safety-centric applications, as well as the high transmission rate requirements for entertainment applications. Consequently, we design a multi-criterion objective reward function that accounts for the heterogeneous traffic demands. Subsequently, we introduce a robust multi-agent proximal policy optimization algorithm with a centralized training and decentralized execution framework. The proposed approach employs global observation to learn policies for each V2V agent during training, while allowing each agent to take an action based on its local observation during distributed execution phase. Furthermore, we consider imperfect spectrum sensing and outdated channel state information (CSI) in the work. Henceforth, a critic network incorporating long short-term memory layers is developed to effectively leverage the global observation, thereby mitigating the impact of outdated CSI and sensing errors. Extensive experiment results demonstrate the superiority of the proposed method comparing to other existing methods in terms of transmission success rate of cellular links, average capacity for entertainment applications of V2V links, and payload delivery rate for safety-centric applications of V2V links. Furthermore, we demonstrate the robustness of the proposed method against variations in V2V payloads and SINR thresholds.

关键词： Resource management optimization Quality of service Sensors Power control Copper Interference Training Wireless networks Vehicle dynamics Deep reinforcement learning heterogeneous applications multi-agent proximal policy optimization resource allocation

来源：评论

学校读者我要写书评

暂无评论

Mobility-aware decentralized federated learning with joint optimization of local iteration and leader selection for vehicular networks

引用

COMPUTER NETWORKS 2025年 263卷

作者： Chen, Dongyu Deng, Tao Jia, Juncheng Feng, Siwei Yuan, Di Soochow Univ Sch Comp Sci & Technol Suzhou Peoples R China Uppsala Univ Dept Informat Technol Uppsala Sweden

Federated learning (FL) emerges as a promising approach to empower vehicular networks, composed by intelligent connected vehicles equipped with advanced sensing, computing, and communication capabilities. While previous studies have explored the application of FL in vehicular networks, they have largely overlooked the intricate challenges arising from the mobility of vehicles and resource constraints. In this paper, we propose a framework of mobility-aware decentralized federated learning (MDFL) for vehicular networks. In this framework, nearby vehicles train an FL model collaboratively, yet in a decentralized manner. We formulate a local iteration and leader selection joint optimization problem (LSOP) to improve the training efficiency of MDFL. For problem solving, we first reformulate LSOP as a decentralized partially observable Markov decision process (Dec-POMDP), and then develop an effective optimization algorithm based on multi-agent proximal policy optimization (MAPPO) to solve Dec-POMDP. Finally, we verify the performance of the proposed algorithm by comparing it with other algorithms.

关键词： Decentralized federated learning Mobility-aware multi-agent proximal policy optimization Vehicular networks

来源：评论

学校读者我要写书评

暂无评论

SpaceRIS: LEO Satellite Coverage Maximization in 6G Sub-THz Networks by MAPPO DRL and Whale optimization

引用

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS 2024年第5期42卷 1262-1278页

作者： Hassan, Sheikh Salman Park, Yu Min Tun, Yan Kyaw Saad, Walid Han, Zhu Hong, Choong Seon Kyung Hee Univ Dept Comp Sci & Engn Yongin 17104 Gyeonggi Do South Korea Aalborg Univ Dept Elect Syst DK-2450 Aalborg Denmark Virginia Tech Bradley Dept Elect & Comp Engn Blacksburg VA 24061 USA Univ Houston Dept Elect & Comp Engn Houston TX 77004 USA Kyung Hee Univ Dept Comp Sci & Engn Yongin 446701 Gyeonggi Do South Korea

Satellite systems face a significant challenge in effectively utilizing limited communication resources to meet the demands of ground network traffic, characterized by asymmetrical spatial distribution and time-varying characteristics. Moreover, the coverage range and signal transmission distance of low Earth orbit (LEO) satellites are restricted by notable propagation attenuation, molecular absorption, and space losses in sub-terahertz (THz) frequencies. This paper introduces a novel approach to maximize LEO satellite coverage by leveraging reconfigurable intelligent surface (RIS) within 6G sub-THz networks. optimization objectives include improving end-to-end (E2E) data rate, optimizing satellite-remote user equipment (RUE) associations, data packet routing within satellite constellations, RIS phase shift, and ground base station (GBS) transmit power (i.e., active beamforming). The formulated joint optimization problem poses significant challenges because of its time-varying environment, non-convex characteristics, and NP-hard complexity. To address these challenges, we propose a block coordinate descent (BCD) algorithm that integrates balanced K-means clustering, multi-agent proximal policy optimization (MAPPO) deep reinforcement learning (DRL), and whale optimization algorithm (WOA) techniques. The performance of the proposed approach is demonstrated through comprehensive simulation results, demonstrating its superiority over existing baseline methods in the literature.

关键词： Satellites Low earth orbit satellites Satellite broadcasting optimization Terahertz communications Routing Array signal processing 6G satellite access networks sub-THz communication reconfigurable intelligent surfaces multi-agent proximal policy optimization whale optimization

来源：评论

学校读者我要写书评

暂无评论

Asynchronous multi-agent Reinforcement Learning-Based Framework for Bi-Level Noncooperative Game-Theoretic Demand Response

引用

IEEE TRANSACTIONS ON SMART GRID 2024年第6期15卷 5622-5637页

作者： Nie, Yongxin Liu, Jun Liu, Xiaoming Zhao, Yu Ren, Kezheng Chen, Chen Xi An Jiao Tong Univ Sch Elect Engn Shaanxi Key Lab Smart Grid Xian 710049 Peoples R China

Demand response (DR) is proposed to address the uncertainty of distributed energy resources (DER) and energy storage systems in distribution networks. However, current research neglects the temporal relationships and the noncooperative relationship of demand response participants. A novel bi-level non-cooperative Stackelberg-Nash dynamic game-theoretic framework is proposed in this paper, specifically designed to address the issue of overlooking consumers' interests and temporal relationships of participant actions in traditional demand response frameworks. Within this innovative framework, an asynchronous multi-agent reinforcement learning algorithm is proposed and named as Stackelberg-Nash multi-agent proximal policy optimization algorithm (SN-MAPPO). SN-MAPPO allows utility company (UC) and consumers to maximize their utilities within the proposed framework, ultimately leading to the convergence of UC and consumers' strategies to Stackelberg-Nash equilibrium. To evaluate the effectiveness of the proposed framework, simulations are conducted using authentic data, involving one UC and eight power consumers. Simulation results confirm that the proposed demand response mechanism, while safeguarding consumers' benefits, fosters the integration of distributed energy resource and effectively mitigates load fluctuations.

关键词： Games Electricity Demand response optimization Smart grids Reinforcement learning Distributed power generation Stackelberg-Nash game asynchronous multi-agent reinforcement learning demand response multi-agent proximal policy optimization

来源：评论

学校读者我要写书评

暂无评论

multi-agent Deep Reinforcement Learning for Recharging-Considered Vehicle Scheduling Problem in Container Terminals

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2024年第11期25卷 16855-16868页

作者： Che, Ada Wang, Ziliang Zhou, Chenhao Northwestern Polytech Univ Sch Management Xian 710072 Peoples R China

The worldwide popularity of electric automated guided vehicles and autonomous vehicles in container terminals requires efficient vehicle scheduling management considering the capacitated battery energy and the charging station limitation. In this paper, the recharging-considered vehicle scheduling problem is formulated as a decentralized partially observable Markov decision process to maximize the cumulative reward, providing a highly adaptive decision-making mechanism for multi-agent-powered terminal transport system by supporting decentralized decisions and accommodating partial observability. Considering the limited number of charging stations and tight schedules, a novel scheduling method based on the actor-critic multi-agent deep reinforcement learning framework is developed to facilitate cooperation among vehicles and charging stations and enhance the stability and efficiency of the learning process. Furthermore, to address the challenge on algorithm convergence due to the vast state space, we employ a heterogeneous graph neural network in the proposed framework for feature extraction and a multi-agent proximal policy optimization algorithm for parameter training. Numerical results indicate that the proposed method outperforms the distributed-agent deep reinforcement learning and several benchmark heuristics, showcasing its superior performance in both solution quality and efficiency. Moreover, the well-trained model can be directly applied to various scenarios, demonstrating its high generalization capability.

关键词： Heuristic algorithms Job shop scheduling Deep reinforcement learning Adaptation models optimization Containers Charging stations Recharging-considered vehicle scheduling multi-agent deep reinforcement learning heterogeneous graph neural network multi-agent proximal policy optimization

来源：评论

学校读者我要写书评

暂无评论

Joint User Pairing and Beamforming Design of multi-STAR-RISs-Aided NOMA in the Indoor Environment via multi-agent Reinforcement Learning

Joint User Pairing and Beamforming Design of Multi-STAR-RISs...

引用

IEEE/IFIP Network Operations and Management Symposium (NOMS)

作者： Park, Yu Min Tun, Yan Kyaw Hong, Choong Seon Kyung Hee Univ Dept Comp Sci & Engn Yongin 17104 South Korea Aalborg Univ Dept Elect Syst AC Meyers Vaenge 15 DK-2450 Aalborg Denmark

ISBN: (纸本)9798350327939;9798350327946

To increase the quality of the 6G / B5G network, conventional cellular networks based on terrestrial base stations are geographically and economically restricted. Meanwhile, Non-Orthogonal multiple Access (NOMA) allows multiple users to share the same resources, which improves the spectral efficiency of the system and has the advantage of supporting a larger number of users. Additionally, by intelligently manipulating the phase and amplitude of both the reflected and transmitted signals, Simultaneously Transmitting and Reflecting RISs (STAR-RISs) can achieve improved coverage, increased spectral efficiency, and enhanced communication reliability. However, STAR-RISs must simultaneously optimize the amplitude and phase shift corresponding to reflection and transmission, which makes existing terrestrial networks more complicated and is considered a major challenge. Motivated by the above, we study the joint user pairing for NOMA and the beamforming design of multi-STAR-RISs in an indoor environment. Then, we formulate the optimization problem with the objective of maximizing the total throughput of mobile users (MUs) by jointly optimizing the decoding order, user pairing, active beamforming, and passive beamforming. However, the formulated problem is a mixed-integer non-linear programming (MINLP). To address this challenge, we first introduce the decoding order for NOMA networks. Next, we decompose the original problem into two subproblems, namely: 1) MU pairing and 2) Beamforming optimization under the optimal decoding order. For the first subproblem, we employ correlation-based K-means clustering to solve the user pairing problem. Then, to jointly deal with beamforming vector optimizations, we propose multi-agent proximal policy optimization (MAPPO), which can make quick decisions in the given environment owing to its low complexity. Finally, simulation results prove that our proposed MAPPO algorithm is superior to proximal policy optimization (PPO) and Advanced Acto

关键词： STAR-RIS NOMA network indoor environment reinforcement learning multi-agent proximal policy optimization

来源：评论

学校读者我要写书评

暂无评论

MAPPO-Based Cooperative UAV Trajectory Design with Long-Range Emergency Communications in Disaster Areas 24

MAPPO-Based Cooperative UAV Trajectory Design with Long-Rang...

引用

24th IEEE International Symposium on a World of Wireless, Mobile and multimedia Networks (WoWMoM)

作者： Guan, Yue Zou, Sai Li, Kai Ni, Wei Wu, Bochun Guizhou Univ Coll Big Data & Informat Engn Guiyang Peoples R China CISTER Res Ctr Porto Portugal CSIRO Canberra ACT Australia Fudan Univ Shanghai Peoples R China

ISBN: (纸本)9798350331653

This paper investigates the cooperative real-time trajectory design issue for multiple unmanned aerial vehicles (UAVs) to support long-range emergency communications in disaster areas. To quickly restore the communication links between mobile users (MUs) and base stations, UAVs equipped with a radio frequency (RF) module and a free space optical (FSO) module serve as relay nodes. Given the difficulty of setting up a central controller for UAVs and the urgency of emergency communication, the UAV trajectory design issue is formulated as a distributed cooperative optimization problem. A collaborative multi-UAV trajectory design method based on multi-agent proximal policy optimization (MAPPO) is adopted to improve RF/FSO channel throughput. Compared with the state-of-the-art DRL methods, MAPPO can achieve higher RF allocation efficiency and increase the FSO communication backhaul capacity.

关键词： unmanned aerial vehicle trajectory optimization long-range free space optical communication radio frequency multi-agent proximal policy optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：