检索结果-内蒙古大学图书馆

distributed q-learning Algorithm for Dynamic Resource Allocation With Unknown Objective Functions and Application to Microgrid

引用

IEEE TRANSACTIONS ON CYBERNETICS 2022年第11期52卷 12340-12350页

作者： Dai, Pengcheng Yu, Wenwu Chen, Duxin Southeast Univ Sch Math Jiangsu Key Lab Networked Collect Intelligence Nanjing 210096 Peoples R China

Dynamic resource allocation problem (DRAP) with unknown cost functions and unknown resource transition functions is studied in this article. The goal of the agents is to minimize the sum of cost functions over given time periods in a distributed way, that is, by only exchanging information with their neighboring agents. First, we propose a distributed q-learning algorithm for DRAP with unknown cost functions and unknown resource transition functions under discrete local feasibility constraints (DLFCs). It is theoretically proved that the joint policy of agents produced by the distributed q-learning algorithm can always provide a feasible allocation (FA), that is, satisfying the constraints at each time period. Then, we also study the DRAP with unknown cost functions and unknown resource transition functions under continuous local feasibility constraints (CLFCs), where a novel distributed q-learning algorithm is proposed based on function approximation and distributed optimization. It should be noted that the update rule of the local policy of each agent can also ensure that the joint policy of agents is an FA at each time period. Such property is of vital importance to execute the epsilon-greedy policy during the whole training process. Finally, simulations are presented to demonstrate the effectiveness of the proposed algorithms.

关键词： Resource management Cost function Approximation algorithms Heuristic algorithms Training distributed algorithms Smart grids distributed optimization distributed q-learning dynamic resource allocation function approximation

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning-Based Online Optimization Algorithm for Unit Commitment and Dispatch in Smart Grid

引用

IEEE TRANSACTIONS ON CYBERNETICS 2020年第9期50卷 4146-4156页

作者： Li, Fangyuan qin, Jiahu Zheng, Wei Xing Univ Sci & Technol China Dept Automat Hefei 230027 Peoples R China Western Sydney Univ Sch Comp Engn & Math Sydney NSW 2751 Australia

Economic dispatch (ED) and unit commitment (UC) problems need to be revisited in order to make a transition from a traditional power system to a smart grid. In this paper, we formulate the ED and UC problems into a unified form, which is also capable of characterizing the infinite horizon UC problem. Based on the formulation, a centralized q-learning-based optimization algorithm is proposed. The proposed algorithm runs in an online manner and requires no prior information on the mathematical formulation of the actual cost functions, thus being capable of dealing with situations for which such cost functions are too difficult to obtain. Then, the distributed counterpart of the centralized algorithm is developed by relaxing the demand for global information and balancing exploration and exploitation cooperatively in a distributed way. Theoretical analysis of the proposed algorithms is also provided. Finally, several case studies are presented to demonstrate the effectiveness of the proposed algorithms.

关键词： distributed q-learning online optimization smart grid unit commitment and dispatch (UCD)

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning Based Joint Relay Selection and Access Control Scheme for IoT-Oriented Satellite Terrestrial Relay Networks

引用

IEEE COMMUNICATIONS LETTERS 2021年第6期25卷 1901-1905页

作者： Zhao, Bo Ren, Guangliang Dong, Xiaodai Zhang, Huining Xidian Univ Sch Telecommun Engn Xian 710071 Peoples R China Northwestern Polytech Univ Sch Cybersecur Xian 710072 Peoples R China Univ Victoria Dept Elect & Comp Engn Victoria BC V8N 5R1 Canada

In this letter, we propose a distributed q-learning (DqL) based joint relay selection and access control (JRSAC) scheme for Internet of Things (IoT)-oriented satellite terrestrial relay networks (STRNs) with massive IoT devices and multiple relays. Firstly, a semi-random access (SRA) architecture is proposed to improve the learning efficiency of the DqL algorithm. Subsequently, a JRSAC optimization problem is formulated and solved by the proposed DqL algorithm. Simulation results show that the proposed DqL based JRSAC scheme significantly outperforms conventional schemes in terms of the medium access control (MAC) throughput, total access delay, and sum rate.

关键词： Relays Satellites Propagation losses Downlink Uplink Delays Resource management Satellite terrestrial relay networks relay selection access control distributed q-learning

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications

引用

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS 2021年第7期39卷 2029-2041页

作者： Liu, Jiajia Shi, Zhenjiang Zhang, Shangwei Kato, Nei Northwestern Polytech Univ Natl Engn Lab Integrated Aerosp Ground Ocean Big Xian 710072 Peoples R China Xidian Univ Sch Cyber Engn State Key Lab Integrated Serv Networks Xian 710071 Peoples R China Tohoku Univ Grad Sch Informat Sci Sendai Miyagi 9808579 Japan

The explosive growth of machine-type communications (MTC) devices poses critical challenges to the existing cellular networks. Therefore, how to support massive MTC devices with limited resources is an urgent problem to be solved. Bursty traffic is an important characteristic of MTC devices, which makes it difficult for agents to learn useful experience and has a negative impact on model convergence. However, most existing reinforcement learning-based literatures assume that devices have saturate data. Towards this end, we propose two distributed q-learning aided uplink grant-free non-orthogonal multiple access (NOMA) schemes (including all-devices distributed q-learning (ADDq) scheme and portion-devices distributed q-learning (PDDq) scheme) to maximize the number of accessible devices, where the bursty traffic of massive MTC devices is carefully considered. In order to reduce the dimension of scheduling space and mitigate the impact of bursty traffic, the idea of grouping devices as well as transmission resources and the intermittent learning mode are adopted in our schemes. Extensive numerical results demonstrate the advantages of proposed schemes from multiple perspectives.

关键词： NOMA Uplink Throughput Convergence Time-frequency analysis Scheduling Resource management Bursty traffic distributed q-learning grant-free non-orthogonal multiple access machine-type communications

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning Aided Heterogeneous Network Association for Energy-Efficient IIoT

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2020年第4期16卷 2756-2764页

作者： Wang, Jingjing Jiang, Chunxiao Zhang, Kai Hou, Xiangwang Ren, Yong qian, Yi Tsinghua Univ Dept Elect Engn Beijing 100084 Peoples R China Tsinghua Univ Tsinghua Space Ctr Beijing 100084 Peoples R China Xidian Univ State Key Lab Integrated Serv Networks Xian 710071 Peoples R China Univ Nebraska Dept Elect & Comp Engn Lincoln NE 68588 USA

To achieve the goal of "Industrial 4.0," cellular network with wide coverage has gradually become an intensely important carrier for industrial Internet of Things (IIoT). The fifth generation cellular network is expected to be a unifying network that may connect billions of IIoT devices for the sake of supporting advanced IIoT business. In order to realize wide and seamless information coverage, heterogeneous network architecture becomes a beneficial method, which can also improve the near-ceiling network capacity. In order to guarantee the quality of service (qoS) as well as the fairness of different IIoT devices with limited network resources, the network association in IIoT should be performed in a more intelligent manner. In this article, we propose a distributed q-learning aided power allocation algorithm for two-layer heterogeneous IIoT networks. Moreover, we discuss the spirit of designing reward functions, followed by four delicately defined reward functions considering both the qoS of femtocell IoT user equipments and macrocell IoT user equipments and their fairness. Also, both fixed and dynamic learning rates and different kinds of multiagent cooperation modes are investigated. Finally, simulation results show the effectiveness and superiority of our proposed q-learning based power allocation algorithm.

关键词： distributed q-learning heterogeneous network industrial Internet of Things (IIoT) power allocation

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning based-Decentralized Resource Allocation for Future Wireless Networks 17

Distributed Q-learning based-Decentralized Resource Allocati...

引用

17th International Multi-Conference on Systems, Signals and Devices (SSD)

作者： Messaoud, Seifeddine Bradai, Abbas Atri, Mohamed Monastir Univ Elect & Microelect Lab Monastir Tunisia Poitiers Univ XLIM Inst Potiers France King Khalid Univ Comp Sci Coll Abha Saudi Arabia

ISBN: (纸本)9781728110806

The next generation (5G) wireless network is expected to support the fourth industrial revolution, combining heightened data transfer speeds and processing power. Despite technological progress, the most important evolution of future networks is the network-densification technique. This technique reduces significantly the propagation distance and makes signal propagation transit from long- to short-range propagation. In this regard, several imposed challenges such as mobility and interference management, and network and resource management can drain the spatial resources and degrade network performance and end user quality of Service (qoS), when the network density exceeds a critical density. We focus in this paper on the resources management and allocation in the context of exploiting the spatial reuse. However, Nodes are authorized to modify their transmission power (TP) and channel according to the resulting throughput. Toward this goal, a distributed q-learning, from the Reinforcement learning category (RL), is proposed to allow nodes to select their TP and the channel only depending on their bit rate rewards. Simulation result proves, after having evaluated the effect of the learning parameters, the effectiveness of the proposed method in finding the best-performing configurations in terms of throughput maximization.

关键词： The Fifth Generation (5G) Reinforcement learning (RL) distributed q-learning The fourth industrial revolution Resources Allocation and Network Densification

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning Based Dynamic Spectrum Management in Cognitive Cellular Systems: Choosing the Right learning Rate 19

Distributed Q-Learning Based Dynamic Spectrum Management in ...

引用

IEEE Symposium on Computers and Communication (ISCC)

作者： Morozs, Nils Clarke, Tim Grace, David Zhao, qiyang Univ York Heslington Dept Elect York YO10 5DD N Yorkshire England

ISBN: (纸本)9781479942763

This paper presents the concept of the Win-or-Learn-Fast (WoLF) variable learning rate for distributed q-learning based dynamic spectrum management algorithms. It demonstrates the importance of choosing the learning rate correctly by simulating a large scale stadium temporary event network. The results show that using the WoLF variable learning rate provides a significant improvement in quality of service, in terms of the probabilities of file blocking and interruption, over typical values of fixed learning rates. The results have also demonstrated that it is possible to provide a better and more robust quality of service using distributed q-learning with a WoLF variable learning rate, than a spectrum sensing based opportunistic spectrum access scheme, but with no spectrum sensing involved.

关键词： Self-Organisation distributed q-learning Dynamic Spectrum Management

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning for Dynamically Decoupled Systems

Distributed Q-Learning for Dynamically Decoupled Systems

引用

American Control Conference (ACC)

作者： Alemzadeh, Siavash Mesbahi, Mehran Univ Washington Dept Aeronaut & Astronaut Seattle WA 98195 USA

ISBN: (纸本)9781538679265

Control of large-scale networked systems often necessitates the availability of complex models for the interactions amongst the agents. However, in many applications building accurate models of these interactions might be prohibitive due to the curse of dimensionality or their inherent complexity. In the meantime, data-guided control methods can circumvent model complexity by directly synthesizing the controller from the observed data. In this paper, we propose a distributed q-learning algorithm to design a feedback mechanism given an underlying graph structure parameterizing the agents' communication. We assume that the distributed nature of the system arises from a common cost and show that for the particular case of identical dynamically decoupled systems, the learned controller converges to the optimal Linear quadratic Regulator controller for each subsystem. We provide a convergence analysis and verify the result with an example.

关键词： distributed q-learning data-guided control linear quadratic regulator networked control systems

来源：评论

学校读者我要写书评

暂无评论

distributed q-learning-Assisted Grant-Free NORA for Massive Machine-Type Communications

Distributed Q-Learning-Assisted Grant-Free NORA for Massive ...

引用

IEEE Global Communications Conference (GLOBECOM) on Advanced Technology for 5G Plus

作者： Shi, Zhenjiang Gao, Wei Liu, Jiajia Kato, Nei Zhang, Yanning Xidian Univ Sch Cyber Engn Xian Peoples R China Huazhong Univ Sci & Technol Sch Elect Informat & Commun Wuhan Peoples R China Northwestern Polytech Univ Sch Cybersecur Xian Peoples R China Tohoku Univ Grad Sch Informat Sci Sendai Miyagi Japan

ISBN: (纸本)9781728182988

Large-scale connectivity support is a critical challenge in the massive machine-type communications scenario. Grant-free random access (RA) is a promising solution because it can reduce severe signaling overhead in contention-based RA procedure. However, there will still be collisions due to the random selection of spectrum resources by the devices. Therefore, we propose a distributed q-learning-assisted grant-free RA scheme to alleviate the collisions between devices. Considering the characteristic of the machine-type communications devices with bursty traffic, the random packet arrival model is adopted in this paper. In order to cope with the difficulties brought by the random transmission of devices to q-learning, an action reward based on the active probabilities of devices is designed. In addition, we introduce the power domain nor-orthogonal multiple access to further enhance the number of accessible devices. Numerical results demonstrate the advantages of the proposed scheme from the devices' successful access probability.

关键词： Machine-type communications grant-free random access distributed q-learning

来源：评论

学校读者我要写书评

暂无评论

RESOURCE ALLOCATION FOR D2D COMMUNICATIONS WITH A NOVEL distributed q-learning ALGORITHM IN HETEROGENEOUS NETWORKS 17

RESOURCE ALLOCATION FOR D2D COMMUNICATIONS WITH A NOVEL DIST...

引用

International Conference on Machine learning and Cybernetics (ICMLC)

作者： Huang, Yung-Fa Tan, Tan-Hsu Wang, Neng-Chung Chen, Young-Long Li, Yu-Ling Chaoyang Univ Technol Dept Informat & Commun Engn Taichung 41368 Taiwan Natl Taipei Univ Technol Dept Elect Engn Taipei Taiwan Natl United Univ Dept Comp Sci & Informat Engn 2 Lienda Miaoli 36063 Taiwan Natl Taichung Univ Sci & Technol Dept Comp Sci & Informat Engn 129Shangmin RdSec 3 Taichung 404 Taiwan

ISBN: (纸本)9781538652145

This study investigates energy efficiency issues of device-to-device (D2D) communications in heterogeneous networks. To minimize the total transmit power, an approach based on q-learning together with logarithmic cooling (LC) is proposed to optimize the connection of user equipment (UE) with base station (BS) or Access point (AP). The proposed distributed q-learning (D-q-learning) with LC can conduct the adequate exploration and exploitation operations for effective optimization. Simulation results indicate that in the multi-cell scenario, the proposed D-q-learning can obtain the sub-optimal performance close to the best solution.

关键词： Device-to-device (D2D) communications Heterogeneous networks q-learning Exploration and exploitation distributed q-learning Logarithmic cooling (LC)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：