检索结果-内蒙古大学图书馆

Age of Information Aware Trajectory Planning of UAVs in Intelligent Transportation Systems: A Deep Learning Approach

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2020年第11期69卷 12382-12395页

作者： Samir, Moataz Assi, Chadi Sharafeddine, Sanaa Ebrahimi, Dariush Ghrayeb, Ali Concordia Univ Montreal PQ H3G 1M8 Canada Lebanese Amer Univ Beirut 11022801 Lebanon Lakehead Univ Thunder Bay ON P7B 5E1 Canada Texas A&M Univ Doha 23874 Qatar

Unmanned aerial vehicles (UAVs) are envisioned to play a key role in intelligent transportation systems to complement the communication infrastructure in future smart cities. UAV-assisted vehicular networking research typically adopts throughput and latency as the main performance metrics. These conventional metrics, however, are not adequate to reflect the freshness of the information, an attribute that has been recently identified as a critical requirement to enable services such as autonomous driving and accident prevention. In this paper, we consider a UAV-assisted single-hop vehicular network, wherein sensors (e.g., LiDARs and cameras) on vehicles generate time sensitive data streams, and UAVs are used to collect and process this data while maintaining a minimum age of information (AoI). We aim to jointly optimize the trajectories of UAVs and find scheduling policies to keep the information fresh under minimum throughput constraints. The formulated optimization problem is shown to be mixed integer non-linear program (MINLP) and generally hard to be solved. Motivated by the success of machine learning (ML) techniques particularly deep learning in solving complex problems with low complexity, we reformulate the trajectories and scheduling policies problem as a Markov decision process (MDP) where the system state space considers the vehicular network dynamics. Then, we develop deep reinforcement learning (DRL) to learn the vehicular environment and its dynamics in order to handle UAVs' trajectory and scheduling policy. In particular, we leverage Deep Deterministic Policy Gradient (DDPG) for learning the trajectories of the deployed UAVs to efficiently minimize the Expected Weighted Sum AoI (EWSA). Simulations results demonstrate the effectiveness of the proposed design and show the deployed UAVs adapt their velocities during the data collection mission in order to minimize the AoI.

关键词： Trajectory Throughput Measurement Roads Machine learning Sensors Unmanned aerial vehicle (UAV) age-of-information scheduling actor-critic algorithm vehicular networks

来源：评论

学校读者我要写书评

暂无评论

actor-critic Based Graphical Games for Discrete-time Linear Systems with Input Constraints

Actor-critic Based Graphical Games for Discrete-time Linear ...

引用

第三十九届中国控制会议

作者： Tian-Xiang Wang Yong Liang Bing-Chang Wang School of Control Science and Engineering Shandong University

In dynamic graphical games, in order to obtain the optimal strategy for each agent, the traditional method is to solve a set of coupled HJB equations. It is very difficult to solve such problems by traditional methods, especially the input of each agent is constrained. actor-critic is a reinforcement learning method that can solve such problems through online iteration. This paper proposes an online iterative algorithm for solving linear discrete-time systems graphics games with input constraints, and this algorithm without the need for drift dynamics of agents. Each agent needs two neural networks to fit the agent’s value function and control strategy, respectively. Finally, a simulation example is given to show the effectiveness of our method.

关键词： actor-critic algorithm Differential Games Input Constraints Neural Network(NN) Reinforcement Learning(RL)

来源：评论

学校读者我要写书评

暂无评论

Manipulator Motion Planning based on actor-critic Reinforcement Learning

Manipulator Motion Planning based on Actor-Critic Reinforcem...

引用

第40届中国控制会议

作者： Qiang Li Jun Nie Haixia Wang Xiao Lu Shibin Song College of Electrical Engineering and Automation Shandong University of Science and Technology

The manipulator control model has the characteristics of high-order,nonlinear,multivariable and strong coupling,which makes it difficult for the manipulator to have good adaptability and *** at the problem of poor reusability and poor autonomy of manipulator applications,a motion planning algorithm based on reinforcement learning is *** this paper,the reinforcement learning continuous control algorithm actor-critic is applied to the motion planning of the manipulator to increase the environmental applicability and autonomy of the manipulator,and realize the intelligent control of the manipulator under simple kinematics *** first,the simulation environment of the hand-eye system of the manipulator is constructed,then the reinforcement learning algorithm model is established according to the simulation environment,and finally,the motion planning training of the manipulator is completed in the simulation *** results demonstrate that the proposed manipulator motion planning algorithm based on actor-critic reinforcement learning has good environmental adaptability and stability.

关键词： Reinforcement Learning actor-critic algorithm Manipulator Motion Planning Value evaluation Policy evaluation

来源：评论

学校读者我要写书评

暂无评论

Leveraging UAVs for Coverage in Cell-Free Vehicular Networks: A Deep Reinforcement Learning Approach

引用

IEEE TRANSACTIONS ON MOBILE COMPUTING 2021年第9期20卷 2835-2847页

作者： Samir, Moataz Ebrahimi, Dariush Assi, Chadi Sharafeddine, Sanaa Ghrayeb, Ali Concordia Univ Montreal PQ H3G 1M8 Canada Lakehead Univ Thunder Bay ON P7B 5E1 Canada Lebanese Amer Univ Beirut 11022801 Lebanon Texas A&M Univ Doha 23874 Qatar

The success in transitioning towards smart cities relies on the availability of information and communication technologies that meet the demands of this transformation. The terrestrial infrastructure presents itself as a preeminent component in this change. Unmanned aerial vehicles (UAVs) empowered with artificial intelligence (AI) are expected to become an integral component of future smart cities that provide seamless coverage for vehicles on highways with poor cellular infrastructure. Motivated by the above, in this paper, we introduce UAVs cell-free network for providing coverage to vehicles entering a highway that is not covered by other infrastructure. However, UAVs have limited energy resources and cannot serve the entire highway all the time. Furthermore, the deployed UAVs have insufficient knowledge about the environment (e.g., the vehicles' instantaneous location). Therefore, it is challenging to control a swarm of UAVs to achieve efficient communication coverage. To address these challenges, we formulate the trajectories decisions making as a Markov decision process (MDP) where the system state space considers the vehicular network dynamics. Then, we leverage deep reinforcement learning (DRL) to propose an approach for learning the optimal trajectories of the deployed UAVs to efficiently maximize the vehicular coverage, where we adopt actor-critic algorithm to learn the vehicular environment and its dynamics to handle the complex continuous action space. Finally, simulations results are provided to verify our findings and demonstrate the effectiveness of the proposed design and show that during the mission time, the deployed UAVs adapt their velocities in order to cover the vehicles.

关键词： Trajectory Road transportation Reinforcement learning Vehicle dynamics Aerospace electronics Wireless networks Task analysis UAV coverage deep reinforcement learning UAVs' trajectories drive-thru actor-critic algorithm vehicular networks

来源：评论

学校读者我要写书评

暂无评论

Formation control scheme with reinforcement learning strategy for a group of multiple surface vehicles

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2024年第3期34卷 2252-2279页

作者： Nguyen, Khai Dang, Van Trong Pham, Dinh Duong Dao, Phuong Nam Carnegie Mellon Univ Dept Mech Engn Pittsburgh PA USA Hanoi Univ Sci & Technol Sch Elect & Elect Engn Hanoi Vietnam

This article presents a comprehensive approach to integrate formation tracking control and optimal control for a fleet of multiple surface vehicles (SVs), accounting for both kinematic and dynamic models of each SV agent. The proposed control framework comprises two core components: a high-level displacement-based formation controller and a low-level reinforcement learning (RL)-based optimal control strategy for individual SV agents. The high-level formation control law, employing a modified gradient method, is introduced to guide the SVs in achieving desired formations. Meanwhile, the low-level control structure, featuring time-varying references, incorporates the RL algorithm by transforming the time-varying closed agent system into an equivalent autonomous system. The application of Lyapunov's direct approach, along with the existence of the Bellman function, guarantees the stability and optimality of the proposed design. Through extensive numerical simulations, encompassing various comparisons and scenarios, this study demonstrates the efficacy of the novel formation control strategy for multiple SV agent systems, showcasing its potential for real-world applications.

关键词： actor-critic algorithm formation control multi-agent system (MAS) reinforcement learning (RL) surface vehicle (SV)

来源：评论

学校读者我要写书评

暂无评论

Uncertainty modified policy for multi-agent reinforcement learning

引用

APPLIED INTELLIGENCE 2024年第22期54卷 12020-12034页

作者： Zhao, Xinyu Liu, Jianxiang Wu, Faguo Zhang, Xiao Wang, Guojian Beihang Univ Sch Math Sci 37 Xueyuan Rd Beijing 100190 Peoples R China Beihang Univ Inst Artificial Intelligence 37 Xueyuan Rd Beijing 100190 Peoples R China Beihang Univ Key Lab Math Informat & Behav Semant LMIB Beijing 100191 Peoples R China Zhongguancun Lab Beijing 100194 Peoples R China Beihang Univ Bejing Adv Innovat Ctr Future Blockchain & Privacy Beijing 100191 Peoples R China

Uncertainty in the evolution of opponent behavior creates a non-stationary environment for the agent, reducing the reliability of value estimation and strategy selection while compromising security during the exploration process. Previous studies have developed various uncertainty quantification techniques and designed uncertainty-aware exploration methods for multi-agent reinforcement learning (MARL). However, existing methods have gaps in theoretical research and experimental verification of decoupling uncertainty between opponents and environment, which can decrease learning efficiency and lead to an unstable training process. Due to inaccurate opponent modeling, the agent is vulnerable to harm from opponents, which is undesirable in real-world tasks. To address these issues, this study proposes a novel uncertainty-guided safe exploration strategy for MARL that decouples the two types of uncertainty originating from the environment and opponents. Specifically, we introduce an uncertainty decoupling quantification technique based on a novel variance decomposition method for action-value functions. Furthermore, we present an uncertainty-aware policy optimization mechanism to facilitate safe exploration in MARL. Finally, we propose a new adaptive parameter scaling method to ensure efficient exploration by the agents. Theoretical analysis establishes the proposed approach's convergence rate, and its effectiveness is demonstrated empirically. Extensive experiments on benchmark tasks spanning differential games, multi-agent particle environments, and RoboSumo validate the proposed uncertainty-guided method's significant advantages in attaining higher scores and facilitating safe agent exploration.

关键词： Multi-agent reinforcement learning Uncertainty quantification actor-critic algorithm Opponent modeling

来源：评论

学校读者我要写书评

暂无评论

Fractional-order fuzzy sliding mode control of uncertain nonlinear MIMO systems using fractional-order reinforcement learning

引用

COMPLEX & INTELLIGENT SYSTEMS 2024年第2期10卷 3057-3085页

作者： Mahmoud, Tarek A. El-Hossainy, Mohammad Abo-Zalam, Belal Shalaby, Raafat Menoufia Univ Fac Elect Engn Dept Ind Elect & Control Engn Menoufia 32952 Egypt Nile Univ SESC Res Ctr Sch Engn & Appl Sci MECT Program Giza 12588 Egypt

This paper introduces a novel approach aimed at enhancing the control performance of a specific class of unknown multiple-input and multiple-output nonlinear systems. The proposed method involves the utilization of a fractional-order fuzzy sliding mode controller, which is implemented through online fractional-order reinforcement learning (FOFSMC-FRL). First, the proposed approach leverages two Takagi-Sugeno-Kang (TSK) fuzzy neural network actors. These actors approximate both the equivalent and switch control parts of the sliding mode control. Additionally, a critic TSK fuzzy neural network is employed to approximate the value function of the reinforcement learning process. Second, the FOFSMC-FRL parameters undergo online adaptation using an innovative fractional-order Levenberg-Marquardt learning method. This adaptive mechanism allows the controller to continuously update its parameters based on the system's behavior, optimizing its control strategy accordingly. Third, the stability and convergence of the proposed approach are rigorously examined using Lyapunov theorem. Notably, the proposed structure offers several key advantages as it does not depend on knowledge of the system dynamics, uncertainty bounds, or disturbance characteristics. Moreover, the chattering phenomenon, often associated with sliding mode control, is effectively eliminated without compromising the system's robustness. Finally, a comparative simulation study is conducted to demonstrate the feasibility and superiority of the proposed method over other control methods. Through this comparison, the effectiveness and performance advantages of the approach are validated.

关键词： Fractional-order sliding mode controller TSK-fuzzy system actor-critic algorithm Levenberg Marquardt method Nonlinear systems

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning with dynamic convex risk measures

引用

MATHEMATICAL FINANCE 2024年第2期34卷 557-587页

作者： Coache, Anthony Jaimungal, Sebastian Univ Toronto Dept Stat Sci Toronto ON Canada Univ Oxford Oxford Man Inst Oxford England

We develop an approach for solving time-consistent risk-sensitive stochastic optimization problems using model-free reinforcement learning (RL). Specifically, we assume agents assess the risk of a sequence of random variables using dynamic convex risk measures. We employ a time-consistent dynamic programming principle to determine the value of a particular policy, and develop policy gradient update rules that aid in obtaining optimal policies. We further develop an actor-critic style algorithm using neural networks to optimize over policies. Finally, we demonstrate the performance and flexibility of our approach by applying it to three optimization problems: statistical arbitrage trading strategies, financial hedging, and obstacle avoidance robot control.

关键词： actor-critic algorithm dynamic risk measures financial hedging policy gradient reinforcement learning robot control time-consistency trading strategies

来源：评论

学校读者我要写书评

暂无评论

Scalable and Decentralized algorithms for Anomaly Detection via Learning-Based Controlled Sensing

引用

IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS 2023年 9卷 640-654页

作者： Joseph, Geethu Zhong, Chen Gursoy, M. Cenk Velipasalar, Senem Varshney, Pramod K. Delft Univ Technol Signal Proc Syst Grp NL-2628 Delft Netherlands Syracuse Univ Dept Elect Engn & Comp Sci Syracuse NY 13244 USA

We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. We develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant, decides when to stop taking observations, and declares the decision on anomalous processes. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value while minimizing the delay in decision making. We devise a centralized algorithm where the processes are jointly selected by a common agent as well as a decentralized algorithm where the decision of whether to select a process is made independently for each process. Our algorithms rely on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithms using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithms have computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of these algorithms using numerical experiments by comparing them with state-of-the-art methods.

关键词： Active hypothesis testing deep learning reinforcement learning actor-critic algorithm quickest state estimation sequential decision-making sequential sensing

来源：评论

学校读者我要写书评

暂无评论

Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning

引用

SIAM JOURNAL ON FINANCIAL MATHEMATICS 2023年第4期14卷 1249-1289页

作者： Coache, Anthony Jaimungal, Sebastian Cartea, Alvaro Univ Toronto Dept Stat Sci Toronto ON M5G 1Z5 Canada Oxford Man Inst Quantitat Finance Oxford OX2 6ED England Univ Oxford Oxford Man Inst Quantitat Finance Oxford OX2 9HB England Univ Oxford Math Inst Oxford OX2 9HB England

We propose a novel framework to solve risk-sensitive reinforcement learning problems where the agent optimizes time-consistent dynamic spectral risk measures. Based on the notion of conditional elicitability, our methodology constructs (strictly consistent) scoring functions that are used as penalizers in the estimation procedure. Our contribution is threefold: we (i) devise an efficient approach to estimate a class of dynamic spectral risk measures with deep neural networks, (ii) prove that these dynamic spectral risk measures may be approximated to any arbitrary accuracy using deep neural networks, and (iii) develop a risk-sensitive actor-critic algorithm that uses full episodes and does not require any additional nested transitions. We compare our conceptually improved reinforcement learning algorithm with the nested simulation approach and illustrate its performance in two settings: statistical arbitrage and portfolio allocation on both simulated and real data.

关键词： dynamic risk measures reinforcement learning elicitability consistent scoring functions time-consistency actor-critic algorithm portfolio allocation statistical arbitrage

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：