Robust Coordinated Reinforcement Learning for MAC Design in Sensor Networks

作     者：Nisioti, Elem Thomos, Nikolaos 

作者机构：Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England 

出 版 物：《IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS》 (IEEE通信选题杂志)

年 卷 期：2019年第37卷第10期

页      面：2211-2224页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 

基　　金：European Union Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Research and Innovation Staff Exchange Grant through the project RECENT 

主　　题：Medium access control Q-learning coordination graphs irregular repetition slotted ALOHA wireless sensor networks POMDP max-sum algorithm 

摘      要：In this paper, we propose a medium access control (MAC) design method for wireless sensor networks based on decentralized coordinated reinforcement learning. Our solution maps the MAC resource allocation problem first to a factor graph, and then, based on the dependencies between sensors, transforms it into a coordination graph, on which the max-sum algorithm is employed to find the optimal transmission actions for sensors. We have theoretically analyzed the system and determined the convergence guarantees for decentralized coordinated learning in sensor networks. As part of this analysis, we derive a novel sufficient condition for the convergence of max-sum on graphs with cycles and employ it to render the learning process robust. In addition, we reduce the complexity of applying max-sum to our optimization problem by expressing coordination as a multiple knapsack problem (MKP). The complexity of the proposed solution can be, thus, bounded by the capacities of the MKP. Our simulations reveal the benefits coming from adaptivity and sensors coordination, both inherent in the proposed learning-based MAC.