Dynamic Channel Selection and Transmission Scheduling for Cognitive Radio Networks

作     者：Zhu, Xinyu Huang, Yang Wu, Qihui Zhou, Fuhui Ge, Xiaohu Liu, Yuan 

作者机构：Nanjing Univ Aeronaut & Astronaut Key Lab Dynam Cognit Syst Elect Spectrum Space Minist Ind & Informat Technol Nanjing 210016 Peoples R China Huazhong Univ Sci & Technol Sch Elect Informat & Commun Wuhan 430074 Peoples R China South China Univ Technol Sch Elect & Informat Engn Guangzhou 510641 Peoples R China 

出 版 物：《IEEE INTERNET OF THINGS JOURNAL》 (IEEE Internet Things J.)

年 卷 期：2022年第9卷第23期

页      面：24429-24443页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China [61901216, U2001210, 61827801, 62071223, 62031012] Natural Science Foundation of Jiangsu Province [BK20190400] 

主　　题：Dynamic scheduling Resource management Optimization Approximation algorithms Time-frequency analysis Data communication Sensors Basis function approximation (BFA) cognitive radio network (CRN) mutually embedded Markov decision processes (MDPs) Q-learning reinforcement learning (RL) resource allocation 

摘      要：Cognitive radio networks (CRNs) are expected to be promising techniques for improving the spectrum efficiency of wireless network utility in the squeezed sub-6-GHz frequency bands. Nevertheless, frequency allocation and transmission scheduling for secondary users (SUs) in CRNs suffer from no prior knowledge of other SUs network behaviors or the distribution of the amount of data generated at each SU. As a countermeasure, this article develops a protocol for the joint channel selection and transmission scheduling such that SUs with heterogeneous data transmission demands could be served with limited spectrum resources. Then, we formulate the dynamic optimization of the protocol as mutually embedded Markov decision processes (MDPs). To address the intractable MDPs, Q-learning-based channel selection and transmission scheduling based on reinforcement learning with basis function approximation are, respectively, proposed. It is shown that compared with various baselines, the proposed channel selection algorithm enables each SU to select the best frequency-domain channel that does not interfere with other SUs. In particular, the proposed transmission scheduling algorithm outperforms algorithms based on off-the-shelf approaches, such as Q-learning and Lyapunov optimization, in terms of both energy efficiency and long-term accumulative amount of bits at each SU.