检索结果-内蒙古大学图书馆

Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

学校读者我要写书评

暂无评论

Procedia Computer Science 2020年 177卷 324-329页

作者： Wenshuai Zhao Jorge Peña Queralta Li Qingqing Tomi Westerlund Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged, the probability of malfunctions or errors must be taken into account. We show how wrong discrete actions can significantly affect the collaborative learning effort. In particular, we analyze the effect of having a fraction of agents that might perform the wrong action with a given probability. We study the ability of the system to converge towards a common working policy through the collaborative learning process based on the number of experiences from each of the agents to be aggregated for each policy update, together with the fraction of wrong actions from agents experiencing malfunctions. Our experiments are carried out in a simulation environment using the Atari testbed for the discrete action spaces, and advantage actor-critic (A2C) for the distributed multi-agent training.

关键词： Reinforcement Learning Edge Computing Multi-Agent systems Collaborative Learning RL Deep RL Adversarial RL

Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning

学校读者我要写书评

暂无评论

Towards Closing the Sim-to-Real Gap in Collaborative Multi-R...

International Conference on robotics and Automation Engineering (ICRAE)

作者： Wenshuai Zhao Jorge Peña Queralta Li Qingqing Tomi Westerlund Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

ISBN: (数字)9781728189819

ISBN: (纸本)9781728189826

Current research directions in deep reinforcement learning include bridging the simulation-reality gap, improving sample efficiency of experiences in distributed multi-agent reinforcement learning, together with the development of robust methods against adversarial agents in distributed learning, among many others. In this work, we are particularly interested in analyzing how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems where the operation of the different robots is not necessarily homogeneous. These variations can happen due to sensing mismatches, inherent errors in terms of calibration of the mechanical joints, or simple differences in accuracy. While our results are simulation-based, we introduce the effect of sensing, calibration, and accuracy mismatches in distributed reinforcement learning with proximal policy optimization (PPO). We discuss on how both the different types of perturbances and how the number of agents experiencing those perturbances affect the collaborative learning effort. The simulations are carried out using a Kuka arm model in the Bullet physics engine. This is, to the best of our knowledge, the first work exploring the limitations of PPO in multi-robot systems when considering that different robots might be exposed to different environments where their sensors or actuators have induced errors. With the conclusions of this work, we set the initial point for future work on designing and developing methods to achieve robust reinforcement learning on the presence of real-world perturbances that might differ within a multi-robot system.

关键词： Robots Robot sensing systems Reinforcement learning Task analysis Perturbation methods Collaborative work Collaboration

Ubiquitous distributed deep reinforcement learning at the edge: Analyzing Byzantine agents in discrete action spaces

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhao, Wenshuai Queralta, Jorge Peña Qingqing, Li Westerlund, Tomi Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

关键词： Deep learning

Sim-to-real transfer in deep reinforcement learning for robotics: A survey

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhao, Wenshuai Queralta, Jorge Peña Westerlund, Tomi Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-to-real gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep learning

Sim-to-Real Transfer in Deep Reinforcement Learning for robotics: a Survey

学校读者我要写书评

暂无评论

Sim-to-Real Transfer in Deep Reinforcement Learning for Robo...

IEEE Symposium Series on Computational Intelligence (SSCI)

作者： Wenshuai Zhao Jorge Peña Queralta Tomi Westerlund Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

ISBN: (数字)9781728125473

ISBN: (纸本)9781728125480

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-toreal gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.

关键词： Robots Reinforcement learning Task analysis Robot sensing systems Training Adaptation models Optimization

Towards closing the sim-to-real gap in collaborative multi-robot deep reinforcement learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhao, Wenshuai Queralta, Jorge Peña Qingqing, Li Westerlund, Tomi Turku Intelligent Embedded and Robotic Systems Lab University of Turku Finland

关键词： Deep learning

Autocalibration of a mobile UWB localization system for Ad-Hoc multi-robot deployments in GNSS-denied environments

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Almansa, Carmen Martínez Shule, Wang Queralta, Jorge Peña Westerlund, Tomi Turku Intelligent Embedded and Robotic Systems University of Turku

Ultra-wideband (UWB) wireless technology has seen an increased penetration in the robotics field as a robust localization method in recent years. UWB enables high accuracy distance estimation from time-of-flight measurements of wireless signals, even in non-line-of-sight measurements. UWB-based localization systems have been utilized in various types of GNSS-denied environments for ground or aerial autonomous robots. However, most of the existing solutions rely on a fixed and well-calibrated set of UWB nodes, or anchors, to estimate accurately the position of other mobile nodes, or tags, through multilateration. This limits the applicability of such systems for dynamic and ad-hoc deployments, such as post-disaster scenarios where the UWB anchors could be mounted on mobile robots to aid the navigation of UAVs or other robots. We introduce a collaborative algorithm for online autocalibration of anchor positions, enabling not only ad-hoc deployments but also movable anchors, based on Decawave's DWM1001 UWB module. Compared to the built-in autocalibration process from Decawave, we drastically reduce the amount of calibration time and increase the accuracy at the same time. We provide both experimental measurements and simulation results to demonstrate the usability of this algorithm. Copyright © 2020, The Authors. All rights reserved.

关键词： Ultra-wideband (UWB)

UWB-Based Localization for Multi-UAV systems and Collaborative Heterogeneous Multi-Robot systems

学校读者我要写书评

暂无评论

Procedia Computer Science 2020年 175卷 357-364页

作者： Wang Shule Carmen Martínez Almansa Jorge Peña Queralta Zhuo Zou Tomi Westerlund School of Information Science and Technology Fudan University Shanghai China Turku Intelligent Embedded and Robotic Systems Lab University of Turku Turku Finland

Ultra-wideband technology has emerged in recent years as a robust solution for localization in GNSS denied environments. In particular, its high accuracy when compared to other wireless localization solutions is enabling a wider range of collaborative and multi-robot application scenarios, being able to replace more complex and expensive motion-capture areas for use cases where accuracy in the order of tens of centimeters is sufficient. We present the first survey of UWB-based localization focused on multi-UAV systems and heterogeneous multi-robot systems. We have found that previous literature reviews do not consider in-depth the challenges in both aerial navigation and navigation with multiple robots, but also in terms of heterogeneous multi-robot systems. In particular, this is, to the best of our knowledge, the first survey to review recent advances in UWB-based (i) methods that enable ad-hoc and dynamic deployments; (ii) collaborative localization techniques; and (iii) cooperative sensing and cooperative maneuvers such as UAV docking on mobile platforms. Finally, we also review existing datasets and discuss the potential of this technology for both localization in GNSS-denied environments and collaboration in multi-robot systems.

关键词： UWB robotics UAVs Multi-Robot systems Multi-UAV systems Ultra-wideband Localization GNSS-Denied Environments

End-to-End Design for Self-Reconfigurable Heterogeneous robotic Swarms

学校读者我要写书评

暂无评论

End-to-End Design for Self-Reconfigurable Heterogeneous Robo...

International Conference on Distributed Computing in Sensor systems (DCOSS)

作者： Jorge Peña Queralta Li Qingqing Tuan Nguyen Gia Hong-Linh Truong Tomi Westerlund Turku Intelligent Embedded and Robotic Systems (TIERS) Lab University of Turku Turku Finland Department of Computer Science Aalto University Finland

The following topics are dealt with: Internet of Things; wireless sensor networks; mobile robots; autonomous aerial vehicles; learning (artificial intelligence); control engineering computing; mobile computing; protoc... 详细信息

ISBN: (数字)9781728143514

ISBN: (纸本)9781728198040

关键词： Robot sensing systems Drones Robot kinematics Collaboration Cloud computing