检索结果-内蒙古大学图书馆

The USV Path Planning Based on an Improved DQN Algorithm

学校读者我要写书评

暂无评论

The USV Path Planning Based on an Improved DQN Algorithm

2021 International Conference on Networking, Communications and Information Technology, NetCIT 2021

作者： Huang, Zhijian Lin, Huiqun Zhang, Guichen Shanghai Maritime University Lab of Intelligent Control and Computation Shanghai201306 China

ISBN: (纸本)9781665400701

Research on the path planning algorithm of USV (unmanned surface vessel) has received widespread attention. Among them, the algorithm based on the combination of deep learning and reinforcement learning achieves better performance. However, in the complex water environment, this algorithm is prone to overestimate the action value, which will lead to path planning deviation. In this paper, we propose a Double DQN algorithm to optimize obstacle avoidance and path planning, which based on deep Q network (DQN) algorithm. The Double DQN algorithm will decouple the target Q value's action selection and action evaluation, and index the action value corresponding to the maximum Q value through the current network, then input the selected action value into the target network to calculate the target Q value. As a result, the overestimation is reduced, and path deviations are corrected. The simulation results show that the Double DQN algorithm has a good performance in path planning compared with the DQN algorithm, and indicate the Double DQN algorithm can effectively process complex environmental information and make optimal path planning. © 2021 IEEE.

关键词： Motion planning

The USV Path Planning Based on an Improved DQN Algorithm

学校读者我要写书评

暂无评论

The USV Path Planning Based on an Improved DQN Algorithm

Networking, Communications and Information Technology (NetCIT), International Conference on

作者： Zhijian Huang Huiqun Lin Guichen Zhang Lab of Intelligent Control and Computation Shanghai Maritime University Shanghai China

ISBN: (纸本)9781665400718

Research on the path planning algorithm of USV (unmanned surface vessel) has received widespread attention. Among them, the algorithm based on the combination of deep learning and reinforcement learning achieves better performance. However, in the complex water environment, this algorithm is prone to overestimate the action value, which will lead to path planning deviation. In this paper, we propose a Double DQN algorithm to optimize obstacle avoidance and path planning, which based on deep Q network (DQN) algorithm. The Double DQN algorithm will decouple the target Q value’s action selection and action evaluation, and index the action value corresponding to the maximum Q value through the current network, then input the selected action value into the target network to calculate the target Q value. As a result, the overestimation is reduced, and path deviations are corrected. The simulation results show that the Double DQN algorithm has a good performance in path planning compared with the DQN algorithm, and indicate the Double DQN algorithm can effectively process complex environmental information and make optimal path planning.

关键词： Training Sea surface Heuristic algorithms Simulation Neural networks Reinforcement learning Path planning

Energy-Efficient Reinforcement Learning for Motion Planning of AUV

学校读者我要写书评

暂无评论

Energy-Efficient Reinforcement Learning for Motion Planning ...

IEEE International Conference on Underwater System Technology: Theory and Applications (USYS)

作者： Jiayi Wen Jingwei Zhu Yejin Lin Guichen Zhang Lab of Intelligent Marine Vehicles of DMU Dalian Maritime University Dalian 116033 China School of Marine Electrical Engineering Dalian Maritime University Dalian 116033 China Lab of Intelligent Control and Computation Shanghai Maritime University Shanghai 201306 China

The accuracy of mapping results depends on the Autonomous Underwater Vehicles (AUVs) navigation errors. However, signals are attenuated drastically in water, which makes it difficult for AUVs to receive signals underwater. As a result, traditional navigation methods may become unreliable. In this paper, a terrain-aided navigation method that takes into account distance and energy consumption is proposed, which does not rely on a precise positioning system. Given the complexity of 3D terrain, this paper formulates the problem as a Markov decision process (MDP) and aims to minimise the energy cost function, where a motion planning method based on soft actor-critic (SAC) is formulated. Then a 3D energy consumption calculation method is developed for the AUV which is accurate for each time slot during the training process. Finally, experiments on the Gym platform were carried out to verify the effectiveness of the proposed method.

关键词： Training Energy consumption Three-dimensional displays Navigation Reinforcement learning Markov processes Cost function

Learning Robust Data-based LQG controllers from Noisy Data

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Wenjie Sun, Jian Wang, Gang Bullo, Francesco Chen, Jie The National Key Lab of Autonomous Intelligent Unmanned Systems School of Automation Beijing Institute of Technology Beijing100081 China The Beijing Institute of Technology Chongqing Innovation Center Chongqing401120 China The Mechanical Engineering Department The Center of Control Dynamical Systems and Computation UC Santa Barbara CA93106-5070 United States The Department of Control Science and Engineering Tongji University Shanghai201804 China The National Key Lab of Autonomous Intelligent Unmanned Systems Beijing Institute of Technology Beijing100081 China

This paper addresses the joint state estimation and control problems for unknown linear time-invariant systems subject to both process and measurement noise. The aim is to redesign the linear quadratic Gaussian (LQG) controller based solely on data. The LQG controller comprises a linear quadratic regulator (LQR) and a steady-state Kalman observer;while the data-based LQR design problem has been previously studied, constructing the Kalman gain and the LQG controller from noisy data presents a novel challenge. In this work, a data-based formulation for computing the steady-state Kalman gain is proposed based on semi-definite programming (SDP) using some noise-free input-state-output data. Additionally, a data-based LQG controller is developed, which is shown to be equivalent to the model-based LQG controller. For cases where offline data are corrupted by noise, a robust data-based observer gain is constructed by tackling a relaxed SDP. The proposed controllers are proven to achieve robust global exponential stability (RGES) for state estimation and input-to-state practical stability (ISpS) under standard conditions. Finally, numerical tests are conducted to validate the proposed controllers' correctness and effectiveness. © 2023, CC BY.

关键词： Linear time-invariant system

Tiny noise, big mistakes: adversarial perturbations induce errors in brain–computer interface spellers

学校读者我要写书评

暂无评论

National Science Review 2021年第4期8卷 78-90页

作者： Xiao Zhang Dongrui Wu Lieyun Ding Hanbin Luo Chin-Teng Lin Tzyy-Ping Jung Ricardo Chavarriaga Ministry of Education Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and AutomationHuazhong University of Science and Technology School of Civil Engineering and Mechanics Huazhong University of Science and Technology Centre of Artificial Intelligence Faculty of Engineering and Information Technology University of Technology Sydney Swartz Center for Computational Neuroscience Institute for Neural ComputationUniversity of California San Diego Center for Advanced Neurological Engineering Institute of Engineering in Medicine University of California San Diego ZHAW Data Lab Zürich University of Applied Sciences

An electroencephalogram(EEG)-based brain–computer interface(BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g. amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a *** studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e. they can be severely attacked by adversarial perturbations,which are too tiny to be noticed when added to EEG signals, but can mislead the spellers to spell anything the attacker wants. The consequence could range from merely user frustration to severe misdiagnosis in clinical applications. We hope our research can attract more attention to the security of EEG-based BCI spellers, and more broadly, EEG-based BCIs, which has received little attention before.

关键词： electroencephalogram brain-computer interfaces BCI spellers adversarial examples

Data-driven Self-triggered control via Trajectory Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Wenjie Sun, Jian Wang, Gang Bullo, Francesco Chen, Jie The State Key Lab of Intelligent Control and Decision of Complex Systems The School of Automation Beijing Institute of Technology Beijing100081 China The Beijing Institute of Technology Chongqing Innovation Center Chongqing401120 China The Mechanical Engineering Department The Center of Control Dynamical Systems and Computation UC Santa BarbaraCA93106-5070 United States The Department of Control Science and Engineering Tongji University Shanghai201804 China The State Key Lab of Intelligent Control and Decision of Complex Systems Beijing Institute of Technology Beijing100081 China

Self-triggered control, a well-documented technique for reducing the communication overhead while ensuring desired system performance, is gaining increasing popularity. However, existing methods for self-triggered control require explicit system models that are assumed perfectly known a priori. An end-to-end control paradigm known as data-driven control learns control laws directly from data, and offers a competing alternative to the routine system identification-then-control method. In this context, the present paper puts forth data-driven self-triggered control schemes for unknown linear systems using data collected offline. Specifically, for output feedback control systems, a data-driven model predictive control (MPC) scheme is proposed, which computes a sequence of control inputs while generating a predicted system trajectory. A data-driven self-triggering law is designed using the predicted trajectory, to determine the next triggering time once a new measurement becomes available. For state feedback control systems, instead of capitalizing on MPC to predict the trajectory, a data-fitting problem using the pre-collected input-state data is solved, whose solution is employed to construct the self-triggering mechanism. Both feasibility and stability are established for the proposed self-triggered controllers, which are validated using numerical examples. © 2022, CC BY.

关键词： Model predictive control

Ensemblefollower: A Hybrid Car-Following Framework Based on Hierarchical Planning and Reinforcement Learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Han, Xu Chen, Xianda Zhu, Meixin Cai, Pinlong Zhou, Jianshan Chu, Xiaowen Guangzhou China Guangzhou China Guangdong Provincial Key Lab of Integrated Communication Sensing and Computation for Ubiquitous Internet of Things Guangzhou China Shanghai Artificial Intelligence Laboratory Shanghai China School of Transportation Science and Engineering Beihang University Beijing China State Key Laboratory of Intelligent Transportation Systems Beijing China Beijing Key Laboratory for Cooperative Vehicle Infrastructure Systems & Safety Control Beijing China

Car-following models have made significant contributions to our understanding of longitudinal driving behavior. However, they often exhibit limited accuracy and flexibility, as they cannot fully capture the complexity inherent in car-following processes, or may struggle in unseen scenarios due to their reliance on confined driving skills present in training data. It is worth noting that each car-following model possesses its own strengths and weaknesses depending on specific driving scenarios. Therefore, we propose EnsembleFollower, a hierarchical planning framework for achieving human-like autonomous car-following. The EnsembleFollower framework involves a high-level Reinforcement Learning-based agent responsible for judiciously managing multiple low-level models (such as intelligent Driver Model and Gipps model) according to the current state, either by selecting an appropriate car-following model to perform an action or by allocating different weights across all primitive models. Moreover, we propose a jerk-constrained kinematic model for more convincing microscopic traffic simulations. We evaluate the proposed method based on real-world driving data from the HighD dataset. The experimental results illustrate that EnsembleFollower yields an improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse driving conditions by leveraging the strengths of various car-following models. © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Han, Xu Chen, Xianda Zhu, Meixin Cai, Pinlong Zhou, Jianshan Chu, Xiaowen Guangzhou China Guangzhou China Guangdong Provincial Key Lab of Integrated Communication Sensing and Computation for Ubiquitous Internet of Things China The Shanghai Artificial Intelligence Laboratory Shanghai China State Key Laboratory of Intelligent Transportation Systems Beijing Key Laboratory for Cooperative Vehicle Infrastructure Systems & Safety Control School of Transportation Science and Engineering Beihang University Beijing100191 China

Car-following models have made significant contributions to our understanding of longitudinal driving behavior. However, they often exhibit limited accuracy and flexibility, as they cannot fully capture the complexity inherent in car-following processes, or may falter in unseen scenarios due to their reliance on confined driving skills present in training data. It is worth noting that each car-following model possesses its own strengths and weaknesses depending on specific driving scenarios. Therefore, we propose EnsembleFollower, a hierarchical planning framework for achieving advanced human-like car-following. The EnsembleFollower framework involves a high-level Reinforcement Learning-based agent responsible for judiciously managing multiple low-level car-following models according to the current state, either by selecting an appropriate low-level model to perform an action or by allocating different weights across all low-level components. Moreover, we propose a jerk-constrained kinematic model for more convincing car-following simulations. We evaluate the proposed method based on real-world driving data from the HighD dataset. The experimental results illustrate that EnsembleFollower yields improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse car-following conditions by leveraging the strengths of various low-level models. Copyright © 2023, The Authors. All rights reserved.

关键词： Motion planning

Data-driven resilient predictive control under denial-of-service

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Liu, Wenjie Sun, Jian Wang, Gang Bullo, Francesco Chen, Jie State Key Lab of Intelligent Control and Decision of Complex Systems The School of Automation Beijing Institute of Technology Beijing100081 China State Key Lab of Intelligent Control and Decision of Complex Systems The School of Automation Beijing Institute of Technology Beijing100081 China The Beijing Institute of Technology Chongqing Innovation Center Chongqing401120 China Mechanical Engineering Department The Center of Control Dynamical Systems and Computation UC Santa Barbara CA93106-5070 United States Department of Control Science and Engineering Tongji University Shanghai201804 China State Key Lab of Intelligent Control and Decision of Complex Systems The School of Automation Beijing Institute of Technology Beijing100081 China

The study of resilient control of linear time-invariant (LTI) systems against denial-of-service (DoS) attacks is gaining popularity in emerging cyber-physical applications. In previous works, explicit system models are required to design a predictor-based resilient controller. These models can be either given a priori or obtained through a prior system identification step. Recent research efforts have focused on data-driven control based on pre-collected input-output trajectories (i.e., without explicit system models). In this paper, we take an initial step toward data-driven stabilization of stochastic LTI systems under DoS attacks, and develop a resilient model predictive control (MPC) scheme driven purely by data-dependent conditions. The proposed data-driven control method achieves the same level of resilience as the model-based control method. For example, local input-to-state stability (ISS) is achieved under mild assumptions on the noise and the DoS attacks. To recover global ISS, two modifications are further suggested at the price of reduced resilience against DoS attacks or increased computational complexity. Finally, a numerical example is given to validate the effectiveness of the proposed control method. © 2021, CC BY.

关键词： Model predictive control