检索结果-内蒙古大学图书馆

Multi-Agent deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2020年第8期69卷 8243-8256页

作者： Wu, Tong Zhou, Pan Liu, Kai Yuan, Yali Wang, Xiumin Huang, Huawei Wu, Dapeng Oliver Huazhong Univ Sci & Technol Sch Elect Informat & Commun Engn Wuhan 430074 Peoples R China Huazhong Univ Sci & Technol Hubei Engn Res Ctr Big Data Secur Sch Cyber Sci & Engn Wuhan 430074 Peoples R China Chongqing Univ Coll Comp Sci Chongqing 400040 Peoples R China Gottingen Univ Inst Comp Sci D-37077 Gottingen Germany South China Univ Technol Sch Comp Sci & Engn Guangzhou 510006 Peoples R China Sun Yat Sen Univ Sch Data & Comp Sci Guangzhou 510006 Peoples R China Univ Florida Dept Elect & Comp Engn Gainesville FL 32611 USA

As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional Q-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.

关键词： Traffic light control deep reinforcement learning deep deterministic policy gradient algorithm markov decision process vehicular network

来源：评论

学校读者我要写书评

暂无评论

Optimal control strategy for solid oxide fuel cell-based hybrid energy system using deep reinforcement learning

引用

IET RENEWABLE POWER GENERATION 2022年第5期16卷 912-921页

作者： Chen, Tao Gao, Ciwei Song, Yutong Southeast Univ Sch Elect Engn Nanjing Peoples R China

This paper proposes a self-adaptive control strategy for solid oxide fuel cell (SOFC) based hybrid energy system using deep reinforcement learning (DRL) techniques. Highly efficient use of hydrogen in a hybrid energy system with the aid of SOFC could create a new paradigm of renewable energy ecosystem and a series of operation principles. Instead of modeling the energy system operation decision-making process as an optimization problem, a DRL framework is used to seek the optimal control strategy with consideration of various physical constraints in the SOFC components and hybrid energy system operation. Specifically, a deep deterministic policy gradient (DDPG) algorithm is used to solve the operation problem and provide the optimal policy guiding the control actions of a tubular SOFC stack, which involves various dynamic characteristics besides the electric measurement. The learnt control strategy may not produce the best result every time, but can guarantee the ultimate benefit in a non-deterministic way in the long-term operation.

关键词： power engineering computing renewable energy ecosystem self-adaptive control strategy SOFC components deep learning (artificial intelligence) control actions optimisation deep deterministic policy gradient algorithm learnt control strategy hybrid energy system operation optimal control adaptive control operation problem solid oxide fuel cells solid oxide fuel cell-based hybrid energy system optimal policy optimization problem deep reinforcement learning techniques reinforcement learning decision making tubular SOFC stack optimal control strategy gradient methods energy system operation decision-making process operation principles

来源：评论

学校读者我要写书评

暂无评论

Parametric Dueling DQN- and DDPG-Based Approach for Optimal Operation of Microgrids

引用

PROCESSES 2024年第9期12卷 1822-1822页

作者： Huang, Wei Li, Qing Jiang, Yuan Lu, Xiaoya Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Univ Sci & Technol Beijing Key Lab Ind Proc Knowledge Automat Minist Educ Beijing 100083 Peoples R China

This study is aimed at addressing the problem of optimizing microgrid operations to improve local renewable energy consumption and ensure the stability of multi-energy systems. Microgrids are localized power systems that integrate distributed energy sources, storage, and controllable loads to enhance energy efficiency and reliability. The proposed approach introduces a novel microgrid optimization method that leverages the parameterized Dueling deep Q-Network (Dueling DQN) and deep deterministic policy gradient (DDPG) algorithms. The method employs a parametric hybrid action-space reinforcement learning technique, where the DDPG is utilized to convert discrete actions into continuous action values corresponding to each discrete action, while the Dueling DQN uses the current observation states and these continuous action values to predict the discrete actions that maximize Q-values. This integrated strategy is designed to tackle the co-scheduling challenge in microgrids, enabling them to dynamically select the most favorable control strategies based on their specific states and the actions of other intelligent entities. The ultimate objective is to minimize the overall operational costs of microgrids while ensuring the efficient local consumption of renewable energy and maintaining the stability of multi-energy systems. Simulation experiments were conducted to validate the efficacy and superiority of the proposed method in achieving the optimal microgrid operation, showcasing its potential to improve service quality and reduce operational expenses. Average rewards increased by 30% and 15% compared to the use of the Dueling DQN or DDPG only.

关键词： microgrid deep deterministic policy gradient algorithm deep reinforcement learning dueling DQN

来源：评论

学校读者我要写书评

暂无评论

Tracking interval control for urban rail trains based on safe reinforcement learning

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2024年第PartB期137卷

作者： Lin, Junting Qiu, Xiaohui Li, Maolin Lanzhou Jiaotong Univ Sch Automat & Elect Engn Lanzhou 730070 Peoples R China

In order to solve the problem of controlling the interval between trains in the new train control system, which aims to ensure the safe operation of trains and improve traffic density, the process of managing train speed is treated as a decision-making process. The utilization of Safe Reinforcement Learning is implemented to attain immediate control of the train interval within the train section. Firstly, utilizing vehicle-to-vehicle communication, the train obtains state information about its surroundings. A constrained Markov Decision Process model is created that takes into account the dynamic characteristics of both the leading and tracking trains. Secondly, by integrating the minimal safety distance and the maximum operating efficiency distance, safety and optimality are connected. An augmented Lagrange multiplier method is utilized to design and implement the safe reinforcement learning algorithm. To enhance the convergence speed of the algorithm, a dual-priority system is implemented, classifying and extracting samples based on their varying levels of importance in empirical samples. Ultimately, simulations were performed to examine various train tracking scenarios. The findings demonstrate that, in the same scenarios, this algorithm surpasses both the Lagrange-based deep deterministic policy gradient algorithm and the fixed lambda based deep deterministic policy gradient algorithm. The safety performance has been improved by 30% and 60%, and the optimality performance has been improved by 40% and 30%, respectively. This algorithm, when paired with safety experience prioritized replay, achieves faster convergence compared to the enhanced version. In general, this algorithm exhibits robust suitability for train tracking interval control.

关键词： New train control system Interval train tracking Safe reinforcement learning deep deterministic policy gradient algorithm Augmented Lagrange method

来源：评论

学校读者我要写书评

暂无评论

A sharing deep reinforcement learning method for efficient vehicle platooning control

引用

IET INTELLIGENT TRANSPORT SYSTEMS 2022年第12期16卷 1697-1709页

作者： Lu, Sikai Cai, Yingfeng Chen, Long Wang, Hai Sun, Xiaoqiang Jia, Yunyi Jiangsu Univ Automot Engn Res Inst Zhenjiang 212013 Jiangsu Peoples R China Jiangsu Univ Sch Automot & Traff Engn Zhenjiang Jiangsu Peoples R China Clemson Univ Int Ctr Automot Reseatch CU ICAR Dept Automot Engn Greenville SC USA Zhenjiang City Jiangsu Univ Engn Technol Res Inst Zhenjiang Jiangsu Peoples R China

The combination of reinforcement learning and platooning control has been widely studied, which has been considered to be altruistic on the basis of safety. However, the platooning control method based on reinforcement learning is not mature. This paper proposes a platoon sharing deep deterministic policy gradient algorithm (PSDDPG) for a multi-vehicle network which overcomes the problem of low efficiency of continuous action space exploration. On the basis of the noise of deep deterministic policy gradient (DDPG) algorithm, additional platoon noise is added to enhance the diversity of training samples during exploration and avoid using a single-vehicle data, which could weaken model robustness. The method of using the time sequence information as an input and replay buffer backup is proposed to prevent the problems of insufficient exploration and low sampling efficiency from deteriorating the training effect. The robustness of the proposed algorithm is verified by tests using the Carla simulator. Also, the platoon merging, overtaking, cruising following and obstacle avoidance control of a vehicle platoon is realized under typical traffic flow conditions. The experimental results show that the PSDDPG algorithm can provide an efficient control strategy and thus has the potential to reduce energy consumption and improve road efficiency.

关键词： obstacle avoidance control mobile robots replay buffer backup low sampling efficiency road vehicles efficient vehicle platooning control energy consumption deep learning (artificial intelligence) road efficiency control engineering computing learning (artificial intelligence) vehicle platoon deep deterministic policy gradient algorithm efficient control strategy training samples single-vehicle data platooning control method platoon merging PSDDPG algorithm road traffic sharing deep reinforcement learning method model robustness time sequence information additional platoon noise reinforcement learning insufficient exploration collision avoidance gradient methods multivehicle network traffic engineering computing continuous action space exploration

来源：评论

学校读者我要写书评

暂无评论

deep reinforcement learning based active safety control for distributed drive electric vehicles

引用

IET INTELLIGENT TRANSPORT SYSTEMS 2022年第6期16卷 813-824页

作者： Wei, Hongqian Zhao, Wenqiang Ai, Qiang Zhang, Youtong Huang, Tianyi Beijing Inst Technol Sch Mech Engn Beijing 100081 Peoples R China

Distributed drive electric vehicles are regarded as the promising transportation due to the advanced power flow architecture. Optimizing the yaw motion to enhance vehicle safety is a challenging job. Besides, the nonlinear features in vehicles affect the control accuracy of the yaw motion controllers. To this end, a deep reinforcement learning (DRL) based direct yaw moment control (DYC) strategy is put forward here. Vehicle dynamics can be approximated with the DRL algorithm, which reduces the complex nonlinear solving process. Concretely, the DYC problem is formulated as Markov Decision Process in which the observed signals and external yaw moment are incorporated as the state and action sets. Thereupon, actor-critic network is exhibited to approximate action-value function and policy function for better control performance. Furthermore, to guarantee the continuous solution of external yaw moment, the deep deterministic policy gradient algorithm is employed, in which target and online network parameters are simultaneously trained to maintain learning process stability. The proposed DRL based DYC strategy is verified using the Carsim/Simulink platform under the typical lane change manoeuvres. Numerical test results demonstrate that the proposed DYC strategy outperforms the linear approaches on taking full advantage of understeer features and enhancing the lateral stability, especially under the critical steering manoeuvres.

关键词： steering systems external yaw moment electric vehicles road vehicles DYC problem nonlinear features deep learning (artificial intelligence) control performance control engineering computing Markov decision process optimisation deep deterministic policy gradient algorithm Carsim platform complex nonlinear solving process yaw motion controllers DRL algorithm learning process stability road safety vehicle safety critical steering manoeuvres DYC strategy reinforcement learning deep reinforcement learning distributed drive electric vehicles lateral stability vehicle dynamics stability motion control Markov processes control accuracy active safety control Simulink platform gradient methods direct yaw moment control strategy advanced power flow architecture approximate action-value function traffic engineering computing

来源：评论

学校读者我要写书评

暂无评论

Computational Performance of deep Reinforcement Learning to Find Nash Equilibria (Jan 2023, 10.1007/s10614-022-10351-6)

引用

COMPUTATIONAL ECONOMICS 2024年第2期63卷 577-577页

作者： Graf, Christoph Zobernig, Viktor Schmidt, Johannes Kloeckl, Claude NYU Inst Policy Integr New York NY 10012 USA Stanford Univ Program Energy & Sustainable Dev PESD Stanford CA 94305 USA Univ Nat Resources & Life Sci Inst Sustainable Econ Dev Vienna Austria

We test the performance of deep deterministic policy gradient—a deep reinforcement learning algorithm, able to handle continuous state and action spaces—to find Nash equilibria in a setting where firms compete in offer prices through a uniform price auction. These algorithms are typically considered “model-free” although a large set of parameters is utilized by the algorithm. These parameters may include learning rates, memory buffers, state space dimensioning, normalizations, or noise decay rates, and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. We show that the algorithm also converges in more complex settings with multiple players and different cost structures. Its reliable convergence may make the method a useful tool to studying strategic behavior of firms even in more complex settings.

关键词： Bertrand equilibrium Competition in uniform price auctions deep deterministic policy gradient algorithm DDPG Parameter sensitivity analysis

来源：评论

学校读者我要写书评

暂无评论

Coordination and Control in Multiagent Systems for Enhanced Pursuit-Evasion Game Performance

引用

DYNAMIC GAMES AND APPLICATIONS 2024年 1-22页

作者： Zhuang, Hua Gao, Pengqun Wu, Xiaotong Zhang, Ying Jia, Huayi Hohai Univ Business Sch Nanjing 211100 Jiangsu Peoples R China Jiangsu Open Univ Business Sch Nanjing 210036 Jiangsu Peoples R China Nantong Univ Business Sch Management Sch Nantong 226019 Jiangsu Peoples R China Hong Kong Polytech Univ Sch Hotel & Tourism Management Hong Kong Peoples R China Zhejiang Coll Secur Technol Gen Adm Off Wenzhou 325016 Peoples R China

This study introduces a creative approach called the Hybrid deep Reinforcement Learning Approach (HYDREIL), which aims to improve coordination and control within the multiagent system for pursuit-evasion games. By combining the advanced techniques of DRL, this study can provide a strong foundation for improving the performance of pursuit-evasion games. The proposed HYDREIL combines the Improved deep deterministic policy gradient algorithm (IDDPG) with fuzzy logic and the BFGS algorithm based on the Armijo-Goldstein criterion with the deep Q Neural Network (DQN). Traditional techniques lag in coordinating with complex and robust systems. The combination provides a robust framework that addresses the difficulties of coordinating multiple agents in pursuit of game scenarios. Pursuing-evasion games aim to catch moving objects in constantly changing surroundings, which presents many difficulties for autonomous systems. This study is based on the bio-inspired differential pursuit of the Dog and Sheep game. In this game environment, the proposed HYDREIL-based DDPG is used to overcome the challenges of the traditional DRL models. At the same time, IDQN is used for real-time decision-making and to improve adaptive learning ability. Through both simulated tests and real-world experiments, the study shows the efficacy of HYDREIL. The ability of the HYDREIL is evaluated according to the learning ability based on average escape rates under challenging scenarios of the game. HYDREIL shows significant improvement even in complex game scenarios and improves the escape rate compared to traditional baseline models. Overall, the proposed HYDREIL framework shows a notable development in multiagent systems and provides an effective solution to the problems associated with control and coordination in pursuit of effective detection scenarios.

关键词： Hybrid deep reinforcement learning approach Coordination Control Multiagent systems Pursuit-evasion games deep deterministic policy gradient algorithm deep Q neural network Optimisation

来源：评论

学校读者我要写书评

暂无评论

deep reinforcement learning-based balancing and sequencing approach for mixed model assembly lines

IET COLLABORATIVE INTELLIGENT MANUFACTURING

引用

IET COLLABORATIVE INTELLIGENT MANUFACTURING 2022年第3期4卷 181-193页

作者： Lv Youlong Tan Yuanliang Ray, Zhong Zhang Peng Wang Junliang Zhang Jie Donghua Univ Inst Artificial Intelligence Shanghai 201620 Peoples R China Shanghai Engn Res Ctr Ind Big Data & Intelligent Shanghai Peoples R China Donghua Univ Coll Mech Engn Shanghai Peoples R China Univ Hong Kong Dept Ind & Mfg Syst Engn Hong Kong Peoples R China

A multi-agent iterative optimisation method based on deep reinforcement learning is proposed for the balancing and sequencing problem in mixed model assembly lines. Based on the Markov decision process model for balancing and sequencing, a balancing agent using a deep deterministic policy gradient algorithm, a sequencing agent using an Actor-Critic algorithm, as well as an iterative interaction mechanism between these agents' output solutions are designed for realising the global optimisation of mixed model assembly lines. The exchange of solution information including assembly time and station workload in the iterative interaction realises the coordination of the worker assignment policy at the balancing stage and the production arrangement policy at the sequencing stage for the minimisation of work overload and idle time at stations. Through the comparative experiments with heuristic rules, genetic algorithms, and the original deep reinforcement learning algorithm, the effectiveness of the proposed method is demonstrated and discussed for small-scale instances as well as large-scale ones.

关键词： original deep reinforcement learning algorithm deep deterministic policy gradient algorithm agents assembling balancing stage genetic algorithms solution information including assembly time learning (artificial intelligence) multiagent optimisation sequencing problem iterative interaction mechanism sequencing stage sequencing agent iterative methods gradient methods multi-agent systems balancing agent Markov decision process model control engineering computing Markov processes mixed model assembly lines

来源：评论

学校读者我要写书评

暂无评论

Parameter Design Optimization for DC-DC Power Converters with deep Reinforcement Learning

Parameter Design Optimization for DC-DC Power Converters wit...

引用

14th Annual IEEE Energy Conversion Congress and Exposition (IEEE ECCE) / IEEE-Industry-Applications-Society (IEEE-IAS) Annual Meeting

作者： Tian, Fanghao Cobaleda, Diego Bernal Wouters, Hans Martinez, Wilmar Katholieke Univ Leuven Energy Ville Dept Elect Engn ESAT Genk Belgium

ISBN: (数字)9781728193878

ISBN: (纸本)9781728193878

The deep deterministic policy gradient (DDPG) algorithm, a reinforcement learning (RL) technique which trains an agent to achieve the maximal reward by interacting with the environment, is applied for parameters optimization for a DC-DC power converter. A model for evaluating the efficiency of both semiconductor and magnetic components is developed. An artificial neural network (ANN) is trained by data from Spice simulation to establish a fast and accurate model for semiconductor losses. In the meantime, magnetic design is conducted by considering various core shapes, materials, winding turns, and airgaps. Finally, DDPG is trained by the combined efficiency evaluation model. The results indicate that the ANN model for semiconductor losses is accurate and the optimal design is obtained quickly by the trained DDPG algorithm.

关键词： Artificial intelligence power converters reinforcement learning deep deterministic policy gradient algorithm design automation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：