检索结果-内蒙古大学图书馆

Advancements in UAV Path Planning: A Deep Reinforcement Learning Approach with soft actor-critic for Enhanced Navigation

引用

UNMANNED SYSTEMS 2025年第4期13卷 1065-1084页

作者： Guo, Jingrui Zhou, Guanzhong Huang, Hailong Huang, Chao Hong Kong Polytech Univ Dept Ind & Syst Engn Hung Hom Hong Kong 999077 Peoples R China Hong Kong Polytech Univ Dept Aeronaut & Aviat Engn Hung Hom Hong Kong 999077 Peoples R China

This paper tackles the intricate challenge of autonomous navigation for Unmanned Aerial Vehicles (UAVs) through dynamically changing environments. We focus on a sophisticated Deep Reinforcement Learning (DRL) approach using the soft actor-critic (SAC) algorithm, optimized for UAV path planning within a continuous action space. This methodology leverages environmental image data to enhance the precision of flight maneuvers and effective obstacle avoidance. Our approach, validated through extensive simulations in Gazebo and field tests, demonstrates the algorithm's efficacy in enabling UAVs to adeptly navigate obstacles using depth maps. The study further explores the robustness of the SAC algorithm by comparing it with traditional DRL methods, emphasizing its superior performance in real-world applications. This research contributes significantly to advancing UAV technology, particularly in autonomous motion planning, by integrating cutting-edge machine learning techniques. The video link is: https://***/watch?v=Nd_aMzejNXY.

关键词： Unmanned aerial vehicles (UAVs) reinforcement learning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Intelligent vehicle driving decision-making model based on variational AutoEncoder network and deep reinforcement learning

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 268卷

作者： Wang, Shufeng Wang, Zhengli Wang, Xinkai Liang, Qingwei Meng, Lingyi Shandong Univ Sci & Technol 579 Qianwangang Rd Qingdao 266590 Shandong Peoples R China Zhongtong Bus Holding Co Ltd 261 Huanghe RdEcon Dev Zone Liaocheng 252000 Shandong Peoples R China

In this paper, an end-to-end driving decision-making model is proposed for intelligent vehicle, utilizing a Variational AutoEncoder (VAE) network and Deep Reinforcement Learning to address the challenges in complex and dynamical driving environments. Firstly, the traffic environment image features are extracted by VAE network, which can effectively reduce the amount of data input and improve the learning efficiency. Secondly, the soft actor-critic (SAC) algorithm is improved through the application of TD error value constraints, N-step learning, etc. Then driving risk field and rule constraints are introduced into the improve SAC algorithm. Based on the real-time driving risk field, the skipping frame method can enhance learning efficiency, and the rule constraints can reduce the dangerous actions in the output of the algorithm. In order to verify the effectiveness of the model, in the CARLA simulation platform the models of scenario and algorithm are established, and the simulations are carried out. The results show that using decision-making model built by the proposed algorithm, the average driving distance by the intelligent vehicle has been improved by 91.37%, the average reward value of the task has been increased by 132.04%, the average success rate of the task has been improved by 46.56%, the training time is also significantly reduced. It demonstrated that the proposed decision-making model provides a significant improvement in driving safety and learning efficiency.

关键词： Intelligent Vehicle Driving Decision-making AutoEncoder Network soft actor-critic algorithm Driving risk field Rule constraints

来源：评论

学校读者我要写书评

暂无评论

An optimal dispatch strategy for 5G base stations equipped with battery swapping cabinets

引用

APPLIED ENERGY 2025年 392卷

作者： Qi, Qi Zhang, Deying Hu, Xiang Li, Xiao Qi, Bing North China Elect Power Univ Sch Elect & Elect Engn Beijing 102206 Peoples R China

The escalating deployment of 5G base stations (BSs) and self-service battery swapping cabinets (BSCs) in urban distribution networks has raised concerns regarding electricity consumption and power efficiency due to their significant energy demands and large numbers. To address this challenge, leveraging the idle energy resources in 5G BS and BSC for distribution network dispatch can alleviate their impact on peak-valley differentials and optimize their benefits. Moreover, as BSCs are predominantly situated at communication tower sites, they not only enhance the backup power capacity for communication loads but also share the power supply capacity with 5G BSs. Consequently, coordinating the dispatch of both 5G BS and BSC can result in enhanced cumulative benefits. Therefore, this paper proposes an optimal dispatch strategy for 5G BSs equipped with BSCs. Firstly, a joint dispatch framework is established, where the idle capacity of batteries in 5G BS and BSC responds to time-of-use tariff and demand response signals. Then, the individual and joint dispatchable capabilities of 5G BS and BSC are formulated, considering their energy storage configuration, operational characteristics and service quality constraints. Following this, an optimal dispatch model of the joint system is developed to maximize the daily operational profit for 5G BS and BSC. The soft actor-critic algorithm is then employed to efficiently generate charging and discharging strategies for all dispatchable units, ensuring communication service quality and meeting battery swapping demands. The solution algorithm is trained and tested in a Python simulation environment, with results validating the effectiveness and efficiency of the proposed strategy in ensuring the timely dispatch of 5G BS and BSC while maximizing their economic advantages.

关键词： 5G base station Battery swapping cabinet Optimal dispatch Reinforcement learning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

A deep reinforcement learning approach for joint scheduling of cascade reservoir system

引用

JOURNAL OF HYDROLOGY 2025年 651卷

作者： Luo, Wei Wang, Chao Zhang, Yunhui Zhao, Jianshi Huang, Zhifeng Wang, Jiaqing Zhang, Chu Tsinghua Univ Sch Civil Engn Beijing 100084 Peoples R China China Inst Water Resources & Hydropower Beijing 100048 Peoples R China CHN Energy Dadu River Big Data Serv Co LTD Chengdu 610041 Sichuan Peoples R China

With the increasing requirements on accurately scheduling of cascade hydropower reservoirs, traditional scheduling models are struggling to provide optimal scheduling decisions under uncertain environments. In this study, a deep reinforcement learning approach is proposed to improve the accuracy and efficiency of optimal load allocation and flood management. In this approach, the scheduling optimization problem is firstly transformed into a model-free multi-step decision problem based on Markov Decision Process (MDP). Then, the soft actor-critic (SAC) algorithm is combined with the Evolutionary Hindsight Experience Replay (EVHER) sampling framework to learn the relationship between scheduling policies and power station states. Finally, an offline realtime optimization model is built for the scheduling of cascade hydropower reservoirs. The robustness of the model is evaluated through 300 Monte Carlo simulations. The proposed approach is applied to the PubugouShenxigou-Zhentouba cascade hydropower reservoir system in the Dadu River basin in China, taking into account the uncertainties of the planned grid load and inflow, as well as the final water level control task. Bit- Flipping experiments demonstrate that EVHER outperforms HER with four sampling strategies in both success rate and algorithm convergence speed, with a mean success rate reaching 0.998. Results from multi-objective scheduling indicate that the proposed deep reinforcement learning approach enables precise scheduling of cascade hydropower reservoir system, achieving a total load deviation rate of no more than 3%.

关键词： Joint scheduling Cascade reservoir system Deep reinforcement learning Water level control soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Deep Reinforcement Learning-Driven Collaborative Rounding-Up for Multiple Unmanned Aerial Vehicles in Obstacle Environments

引用

DRONES 2024年第9期8卷 464-464页

作者： Zhao, Zipeng Wan, Yu Chen, Yong Natl Univ Def Technol Lab Big Data & Decis Changsha 410003 Peoples R China Chengdu Fluid Dynam Innovat Ctr 75 West 2nd Sect2nd Ring Rd Chengdu 610071 Peoples R China Sun Yat Sen Univ Sch Syst Sci & Engn Guangzhou 510275 Peoples R China

With the rapid advancement of UAV technology, the utilization of multi-UAV cooperative operations has become increasingly prevalent in various domains, including military and civilian applications. However, achieving efficient coordinated rounding-up of multiple UAVs remains a challenging problem. This paper addresses the issue of collaborative drone hunting by proposing a decision-making control model based on deep reinforcement learning. Additionally, a shared experience data pool is established to facilitate communication between drones. Each drone possesses independent decision-making and control capabilities while also considering the presence of other drones in the environment to collaboratively accomplish obstacle avoidance and rounding-up tasks. Furthermore, we redefine and design the reward function of reinforcement learning to achieve precise control of drone swarms in diverse environments. Simulation experiments demonstrate the feasibility of the proposed method, showcasing its successful completion of obstacle avoidance, tracking, and rounding-up tasks in an obstacle environment.

关键词： multi-UAV obstacle avoidance rounding-up multi-agent deep reinforcement learning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Optimal Operable Power Flow: Sample-Efficient Holomorphic Embedding-Based Reinforcement Learning

引用

IEEE TRANSACTIONS ON POWER SYSTEMS 2024年第1期39卷 1739-1751页

作者： Sayed, Ahmed Rabee Zhang, Xian Wang, Guibin Wang, Cheng Qiu, Jing Harbin Inst Technol Sch Mech Engn & Automat Shenzhen 518055 Peoples R China Cairo Univ Elect Power & Machines Dept Fac Engn Giza 12411 Egypt Shenzhen Univ Coll Mechatron & Control Engn Shenzhen 518060 Peoples R China North China Elect Power Univ Sch Elect & Elect Engn Beijing 102206 Peoples R China Univ Sydney Sch Elect & Informat Engn Sydney NSW 2006 Australia

The nonlinearity of physical power flow equations divides the decision-making space into operable and non-operable regions. Therefore, existing control techniques could be attracted to non-operable mathematically-feasible decisions. Moreover, the raising uncertainties of modern power systems need quick-optimal actions to maintain system security and stability. This paper proposes a holomorphic embedding-based soft actor-critic (HE-SAC) algorithm to find fast optimal operable power flow (OOPF) by leveraging deep reinforcement learning and advanced complex analysis techniques. First, a dynamic HE-based layer is developed to guarantee the solution operability and uses the previous operable germ instead of the no-load germ for high computational efficiency. Second, a model-based policy optimization is built based on a novel predictive model to generate more data and raise the sample efficiency of the SAC algorithm. Third, the reward function is augmented with the degree of constraint violations and policy entropy to enhance the solution's feasibility. Simulation results demonstrate the computational performance of the proposed dynamic HE layer, and the surpassing of the proposed HE-SAC over a number of state-of-the-art model-free RL algorithms and optimization methods. The proposed approach indicates its practicability and fast operable control for power system operation.

关键词： Reinforcement learning holomorphic embedding operable power flow soft actor-critic algorithm model-based policy optimization

来源：评论

学校读者我要写书评

暂无评论

Enhanced hierarchical reinforcement learning for co-optimization of HVAC system operations

引用

JOURNAL OF BUILDING ENGINEERING 2025年 106卷

作者： Zhou, Xuan Li, Junqiang Mo, Haohua Yan, Junwei Liang, Liequan Pan, Dongmei South China Univ Technol Sch Mech & Automot Engn 381 Wushan Rd Guangzhou 510640 Peoples R China South China Univ Technol Guangzhou Inst Modern Ind Technol Guangzhou 510640 Peoples R China Guangdong Artificial Intelligence & Digital Econ L Guangzhou 510330 Peoples R China Guangdong Univ Finance & Econ Sch Informat Sci Guangzhou 510220 Peoples R China

In the future, building energy consumption is expected to account for approximately 50 % of global energy use, with central air conditioning systems contributing to over 40 % of this consumption. Therefore, enhancing the efficiency of air-conditioning systems is critical. Central air conditioning systems are complex, involving multiple units with varying operating parameters, influenced by factors such as outdoor temperature and occupancy. In real-world operations, the number of equipment units must be dynamically adjusted to accommodate fluctuations in cooling load, while parameters need to be optimized according to equipment configurations. Most existing studies focus on optimizing parameters under a fixed number of units. The collaborative optimization of equipment numbers and parameters across multiple time scales remains a challenge. To address this, we propose an improved hierarchical soft actor-critic (H-SAC) algorithm. This algorithm integrates option-critic with deep reinforcement learning, using Double QNetwork (DDQN) to mitigate overestimation and maximum entropy policies to enhance exploration. By structuring the problem hierarchically and considering environmental factors, the highlevel agent determines the number of equipment units, while the low-level agent optimizes the parameters, enabling collaborative optimization under dynamic conditions. We validated our approach through TRNSYS simulations. Results indicate that, under the premise that the average indoor comfortable time increased by 36.02 %, 30.41 %, 28.44 %, 56.56 %, 17.95 %, 1.93 %, 0.68 %, and 0.30 % respectively, the system energy consumption based on the improved H-SAC algorithm decreased by 13.86 %, 18.14 %, 12.81 %, 22.11 %, 5.57 %, 5.72 %, 2.21 %, and 2.07 % compared to the other eight conventional DRL and DHRL methods, respectively, with faster convergence and enhanced learning capabilities. This research offers an effective solution for HVAC energy-saving optimization, unlocking addit

关键词： tems Deep hierarchical reinforcement learning Model-free control soft actor-critic algorithm TRNSYS simulation platform

来源：评论

学校读者我要写书评

暂无评论

CONTROLLED SENSING AND ANOMALY DETECTION VIA soft actor-critic REINFORCEMENT LEARNING 47

CONTROLLED SENSING AND ANOMALY DETECTION VIA SOFT ACTOR-CRIT...

引用

47th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Zhong, Chen Gursoy, M. Cenk Velipasalar, Senem Syracuse Univ Dept Elect Engn & Comp Sci Syracuse NY 13244 USA

ISBN: (纸本)9781665405409

To address the anomaly detection problem in the presence of noisy observations and to tackle the tuning and efficient exploration challenges that arise in deep reinforcement learning algorithms, we in this paper propose a soft actor-critic deep reinforcement learning framework. To evaluate the proposed framework, we measure its performance in terms of detection accuracy, stopping time, and the total number of samples needed for detection. Via simulation results, we demonstrate the performance when soft actor-critic algorithms are employed, and identify the impact of key parameters, such as the sensing cost, on the performance. In all results, we further provide comparisons between the performances of the proposed soft actor-critic and conventional actor-critic algorithms.

关键词： Anomaly detection controlled sensing reinforcement learning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Cascaded Deep Reinforcement Learning-Based Multi-Revolution Low-Thrust Spacecraft Orbit-Transfer

引用

IEEE ACCESS 2023年 11卷 82894-82911页

作者： Zaidi, Syed Muhammad Talha Chadalavada, Pardha Sai Ullah, Hayat Munir, Arslan Dutta, Atri Kansas State Univ Dept Comp Sci Manhattan KS 66506 USA Wichita State Univ Dept Aerosp Engn Wichita KS 67260 USA

Transferring an all-electric spacecraft from a launch injection orbit to the geosynchronous equatorial orbit (GEO) using a low thrust propulsion system presents a significant challenge due to the long transfer time typically spanning several months. To address the challenge of determining such long time-scale orbit-raising maneuvers to GEO, this paper presents a novel technique to compute transfers starting from geostationary transfer orbit (GTO) and super-GTO. The transfer is complex, involving multiple eclipses and revolutions. To tackle this challenge, we introduce a cascaded deep reinforcement learning (DRL) model to guide a low-thrust spacecraft towards the desired orbit by determining an appropriate thrust direction at each state. To ensure mission requirements, a gradient-aided reward function incorporating the orbital elements, guides the DRL agent to obtain the optimal flight time. The obtained results demonstrate that our proposed approach yields optimal or near-optimal time-efficient spacecraft orbit-raising. DRL implementation is important for spacecraft autonomy;in this context, we demonstrate that our DRL-based trajectory planning provides significantly better transfer time as compared to state-of-the-art approaches that allowfor automated trajectory computation.

关键词： ~Deep reinforcement learning cascaded reinforcement learning soft actor-critic algorithm spacecraft orbit-transfer solar-electric propulsion optimization low-thrust orbit-raising

来源：评论

学校读者我要写书评

暂无评论

Feasibility Constrained Online Calculation for Real-Time Optimal Power Flow: A Convex Constrained Deep Reinforcement Learning Approach

引用

IEEE TRANSACTIONS ON POWER SYSTEMS 2023年第6期38卷 5215-5227页

作者： Sayed, Ahmed Rabee Wang, Cheng Anis, Hussein I. Bi, Tianshu North China Elect Power Univ Sch Elect & Elect Engn Beijing 102206 Peoples R China Cairo Univ Fac Engn Elect Power & Machines Dept Giza 12411 Egypt

Due to the increasing uncertainties of renewable energy and stochastic demands, quick-optimal control actions are necessary to retain the system stability and economic operation. Existing optimal power flow (OPF) solution methods need to be enhanced to guarantee the solution optimality and feasibility in real-time operation under such uncertainties. This paper proposes a convex constrained soft actor-critic (CC-SAC) deep reinforcement learning (DRL) algorithm for the AC-OPF problem. First, this problem is standardized as a Markov decision process model to be solved by DRL algorithms. Second, the operational constraints are satisfied by a novel convex safety layer based on the penalty convex-concave procedure (P-CCP). Then, the control policy is updated by the state-of-the-art off-policy entropy maximization-based SAC algorithm. Therefore, the CC-SAC is a combination of data-driven and physics-driven approaches. The former speedups the solution time by predicting near-optimum control actions through a deep neural network. The latter effectively guarantees the solution feasibility. Simulation results demonstrate the computational performance of the proposed CC-SAC to effectively find AC-OPF decisions with no constraint violation, zero optimality gap and high speed up to 34 times compared to a state-of-the-art solver. The proposed approach indicates its practicability for power system real-time operation and marketing.

关键词： Difference-of-convex programming optimal power flow reinforcement learning safe exploration soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：