检索结果-内蒙古大学图书馆

JOURNAL OF NONLINEAR FUNCTIONAL ANALYSIS 2025年 2025卷

作者： Gao, Xiaonan Wu, Ziyi Zhu, Xianchao Cai, Lei Henan Univ Technol Sch Artificial Intelligence & Big Data Zhengzhou 450001 Peoples R China Henan Inst Sci & Technol Sch Artificial Intelligence Xinxiang 453003 Peoples R China

In recent years, breakthroughs were made in the field of deep reinforcement learning, but, their applications in the real world were seriously affected due to the instability of algorithms and the difficulty in ensuring convergence. As a typical algorithm in reinforcement learning, although the SAC algorithm enhances the robustness and agent's exploration ability by introducing the concept of maximum entropy, it still has the disadvantage of instability in the training process. In order to solve the problems, this paper proposes an Adaptive Normalization-based SAC (AN-SAC) algorithm. By introducing the adaptive normalized reward mechanism into the SAC algorithm, our method can dynamically adjust the normalized parameters of the reward during the training process so that the reward value has zero mean and unit variance. Thus it better adapts to the reward distribution and improves the performance and stability of the algorithm. Experimental results demonstrate that the performance and stability of the AN-SAC algorithm are significantly improved compared with the SAC algorithm.

关键词： Adaptive normalization Deep reinforcement learning Reward mechanism soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

soft actor-critic algorithm Featured Residential Demand Response Strategic Bidding for Load Aggregators

引用

IEEE TRANSACTIONS ON INDUSTRY APPLICATIONS 2022年第4期58卷 4298-4308页

作者： Zhang, Zhenyuan Chen, Zihan Lee, Wei-Jen Univ Elect Sci & Technol China Chengdu 611731 Peoples R China Univ Texas Arlington Arlington TX 76019 USA

Load aggregators (LAs) gather regional demand response (DR) resources as a representative of regional users to transact with the grid. They achieve this by facilitating residential customers to flexibly participate in DR programs. This article proposes a residential DR strategic bidding approach for LAs to improve system reliability while maximizing their financial advantages. It is based on the deep reinforcement learning (DRL) algorithm and is capable of overcoming the computational limitations of conventional model-based strategy design. A probabilistic model considering the electricity usage habits and response willingness was developed to reliably quantify the uncertainty responsive behaviors. A DR bidding model considering bilateral interaction with declaration risk was developed to balance the contradictory relationship between subsidy prices with uncertain DR quantities. In addition, the soft actor-critic algorithm was implemented to optimize the decision-making process, and thereby, optimize the continuous-action-space problem of maximizing the profits of LAs by DR scheduling. The proposed approach demonstrated its effectiveness and superiority in diverse DR situations, compared with a conventional method and another DRL approach.

关键词： Load modeling Probabilistic logic Optimization Uncertainty Decision making Predictive models Demand response Bidding strategy demand response (DR) load aggregators (LAs) power market soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Optimal Power Dispatch of Active Distribution Network and P2P Energy Trading Based on soft actor-critic algorithm Incorporating Distributed Trading Control

引用

JOURNAL OF MODERN POWER SYSTEMS AND CLEAN ENERGY 2025年第2期13卷 540-551页

作者： Zhang, Yongjun Zhang, Jun Wu, Guangbin Zheng, Jiehui Liu, Dongming An, Yuzheng South China Univ Technol Sch Elect Power Guangdong Key Lab Clean Energy Technol Guangzhou 510641 Peoples R China Guangdong Power Grid Corp Customer Serv Ctr Foshan Peoples R China

Peer-to-peer (P2P) energy trading in active distribution networks (ADNs) plays a pivotal role in promoting the efficient consumption of renewable energy sources. However, it is challenging to effectively coordinate the power dispatch of ADNs and P2P energy trading while preserving the privacy of different physical interests. Hence, this paper proposes a soft actor-critic algorithm incorporating distributed trading control (SAC-DTC) to tackle the optimal power dispatch of ADNs and the P2P energy trading considering privacy preservation among prosumers. First, the soft actor-critic (SAC) algorithm is used to optimize the control strategy of device in ADNs to minimize the operation cost, and the primary environmental information of the ADN at this point is published to prosumers. Then, a distributed generalized fast dual ascent method is used to iterate the trading process of prosumers and maximize their revenues. Subsequently, the results of trading are encrypted based on the differential privacy technique and returned to the ADN. Finally, the social welfare value consisting of ADN operation cost and P2P market revenue is utilized as a reward value to update network parameters and control strategies of the deep reinforcement learning. Simulation results show that the proposed SAC-DTC algorithm reduces the ADN operation cost, boosts the P2P market revenue, maximizes the social welfare, and exhibits high computational accuracy, demonstrating its practical application to the operation of power systems and power markets.

关键词： Costs Distribution networks Privacy Peer-to-peer computing Optimization Regulation Electricity Optimal power dispatch peer-to-peer (P2P) energy trading active distribution network (ADN) distributed trading soft actor-critic algorithm privacy preservation

来源：评论

学校读者我要写书评

暂无评论

Joint Access Points-User Association and Caching Placement Strategy for Cell-Free Massive MIMO Systems Based on soft actor-critic algorithm

引用

IEEE COMMUNICATIONS LETTERS 2024年第2期28卷 347-351页

作者： Wang, Rui Shen, Min He, Yun Liu, Xiangyan Chongqing Univ Posts & Telecommun Sch Commun & Informat Engn Chongqing 400065 Peoples R China Yuxi Normal Univ Sch Phys & Elect Engn Yuxi 653100 Yunnan Peoples R China

The joint optimization of APs-user association (AUA) and caching placement (CP) presents a promising method to mitigate the fronthaul overhead and reduce user-perceived latency in cell-free (CF) massive MIMO systems. However, previous studies have not considered the impact of deploying multiple access points (APs) and the user-centric nature of CF massive MIMO systems on joint schemes, while also lacking flexibility in managing caching and association resources. In this letter, we initially develop a latency model based on the collaborative caching and user-centric characteristic. We then formulate a long-term joint optimization problem for AUA and CP to minimize content delivery latency. A two-timescale soft actor-critic (SAC) deep reinforcement learning based algorithm is proposed to solve this NP-hard problem. To enhance the convergence of the SAC algorithm, we devise user clustering and cache partitioning schemes. Numerical results demonstrate that our proposed scheme exhibits reduced latency compared to the benchmark schemes, while efficiently utilizing fronthaul and caching resources.

关键词： Massive MIMO Collaboration Clustering algorithms Wireless communication Partitioning algorithms Optimization Telecommunications APs-user association caching placement user clustering cache partitioning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Self-learning adaptive power management scheme for energy-efficient IoT-MEC systems using soft actor-critic algorithm

INTERNET OF THINGS

引用

INTERNET OF THINGS 2025年 31卷

作者： Rahmani, Amir Masoud Haider, Amir Moghaddasi, Komeil Gharehchopogh, Farhad Soleimanian Aurangzeb, Khursheed Liu, Zhe Hosseinzadeh, Mehdi Natl Yunlin Univ Sci & Technol Future Technol Res Ctr Yunlin Taiwan Sejong Univ Dept AI & Robot Seoul 05006 South Korea Islamic Azad Univ Dept Comp Engn Urmia Branch Orumiyeh Iran King Saud Univ Coll Comp & Informat Sci Dept Comp Engn POB 51178 Riyadh 11543 Saudi Arabia Xinyu Univ Coll Math & Comp Xinyu 338004 Peoples R China Univ Sains Malaysia Sch Comp Sci George Town 11800 Penang Malaysia Duy Tan Univ Sch Comp Sci Da Nang Vietnam Jadara Univ Jadara Res Ctr Irbid 21110 Jordan

The rapid increase of Internet of Things (IoT) devices in Mobile Edge Computing (MEC) environments requires effective energy management to ensure device operation and enhance network efficiency. IoT-MEC systems face challenges such as varying task loads, dynamic environmental conditions, and limited energy resources. These factors make it challenging to design adaptive and efficient energy strategies. Traditional methods, such as static scheduling and centralized control strategies, struggle to adapt to real-time fluctuations in task loads and network conditions, resulting in inefficient energy use, higher latency, and a lack of flexibility to respond to these demands in real-time. This paper proposes a self-learning power management model using the soft actor-critic (SAC) algorithm. It is deployed on IoT devices to enable localized and contextaware power management. Our model includes modules for energy monitoring, adaptive task prioritization, and a self-adjusting reinforcement-learning mechanism, which dynamically finetunes energy policies based on real-time device conditions, allowing each device to optimize power use independently without heavy dependence on centralized control. MEC nodes gather data on battery health, load, and network conditions to support decentralized policy adjustments. Connected devices in simulated smart homes served as the primary context for evaluation. Experimental results show that our model achieves a 45 % reduction in energy consumption in smart home environments, a 49 % improvement in battery life (compared to baseline-like Local Computing), and high adaptability in diverse scenarios compared with other methods.

关键词： Energy management Internet of things soft actor-critic algorithm Context-aware optimization

来源：评论

学校读者我要写书评

暂无评论

Optimal scheduling of virtual power plant based on soft actor-critic algorithm 6

Optimal scheduling of virtual power plant based on Soft Acto...

引用

6th Asia Energy and Electrical Engineering Symposium (AEEES)

作者： Pan, Pengfei Song, Minggang Zou, Nan Qin, Junhan Li, Guangdi Ma, Hongyuan Dalian Power Supply Co Ltd State Grid Liaoning Elect Power Supply Co Ltd Dalian Peoples R China Northeastern Univ Coll Informat Sci & Engn Shenyang Peoples R China

ISBN: (纸本)9798350373486;9798350373479

Aiming at the problem that distributed power sources are difficult to be centrally scheduled and optimized, a virtual power plant (VPP) optimization scheduling method based on deep reinforcement learning is proposed. The method can adaptively adapt to the uncertainty of source load and give real-time optimal scheduling strategies in the presence of source load fluctuations. First, the system framework of VPP is constructed, and the optimal scheduling model of VPP is established with the objective function of minimum daily operating cost, and various operating condition constraints are practically considered. Then, the basic principle and process of deep reinforcement learning are introduced, and the constructed VPP optimal scheduling model is transformed into a deep reinforcement learning model, and the state space, action space and reward function are reasonably designed. Following that, soft actor-critic algorithm (SAC) is used for offline training, and the trained model is utilized for online decision making. Finally, the simulation is carried out in an area as an example, and the results show that the VPP optimal scheduling strategy proposed in this paper can effectively reduce the operating cost of VPP and ensure the safe operation of the grid after grid connection.

关键词： Virtual power plant deep reinforcement learning optimal scheduling soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

Energy optimization management of microgrid using improved soft actor-critic algorithm

引用

INTERNATIONAL JOURNAL OF RENEWABLE ENERGY DEVELOPMENT-IJRED 2024年第2期13卷 329-339页

作者： Yu, Zhiwen Zheng, Wenjie Zeng, Kaiwen Zhao, Ruifeng Zhang, Yanxu Zeng, Mengdi Elect Power Dispatching & Control Ctr Guangdong Po Guangzhou 510600 Peoples R China China Southern Power Grid Technol Co Ltd Guangdong Prov Key Lab Smart Grid New Technol Ente Guangzhou 510180 Peoples R China

To tackle the challenges associated with variability and uncertainty in distributed power generation, as well as the complexities of solving high -dimensional energy management mathematical models in mi-crogrid energy optimization, a microgrid energy optimization management method is proposed based on an improved soft actor -critic algorithm. In the proposed method, the improved soft actor -critic algorithm employs an entropybased objective function to encourage target exploration without assigning signifi-cantly higher probabilities to any part of the action space, which can simplify the analysis process of distributed power generation variability and uncertainty while effectively mitigating the convergence fragility issues in solving the high -dimensional mathematical model of microgrid energy management. The effectiveness of the proposed method is validated through a case study analysis of microgrid energy op-timization management. The results revealed an increase of 51.20%, 52.38%, 13.43%, 16.50%, 58.26%, and 36.33% in the total profits of a microgrid compared with the Deep Q -network algorithm, the state -action -reward -state -action algorithm, the proximal policy optimization algorithm, the ant -colony based algorithm, a microgrid energy optimization management strategy based on the genetic algorithm and the fuzzy inference system, and the theoretical retailer stragety, respectively. Additionally, com-pared with other methods and strategies, the proposed method can learn more optimal microgrid energy management behaviors and anticipate fluctuations in electricity prices and demand.

关键词： Energy optimization management Electricity rate Microgrid Reinforcement learning soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

The soft actor-critic algorithm for automatic mode-locked fiber lasers

引用

OPTICAL FIBER TECHNOLOGY 2023年 81卷

作者： Li, Jin Chang, Kun Liu, Congcong Ning, Yu Ma, Yuansheng He, Jiangyong Liu, Yange Wang, Zhi Nankai Univ Inst Modern Opt Tianjin Key Lab Microscale Opt Informat Sci & Tech Tianjin 300350 Peoples R China

With the development of artificial intelligence, deep reinforcement learning (DRL) has been applied for fiber laser. In this paper, the intelligent passively mode-locked fiber laser (PMLFL) with the soft actor-critic (SAC) algorithm is reported. The SAC algorithm is a DRL algorithm with random policy, which combines the actor-critic framework and the maximum entropy. The agent learns the logic of mode-locking by outputting actions and inputting states of the laser. Due to the maximum entropy model, more exploration is encouraged, which means that multiple policies can be learned to maximize the reward, and the robustness is enhanced accordingly. The results show that the logic learned by the agent is similar to that of human. In 80 random initial state of polarization mode-locked tests, 37 explorations are needed on average, and the frequency of achieving mode-locked state exceeds 0.8 within 60 explorations. Further, the laser system can be monitored or controlled remotely, which expands the application scenarios.

关键词： Fiber laser Automatic mode-locking Deep reinforcement learning soft actor-critic algorithm Passively mode-locked fiber laser

来源：评论

学校读者我要写书评

暂无评论

Residential Demand Response Considered Strategic Bidding for Load Aggregators with soft actor-critic algorithm

Residential Demand Response Considered Strategic Bidding for...

引用

Annual Meeting of the IEEE-Industry-Applications-Society (IAS)

作者： Zhang, Zhenyuan Chen, Zihan Lee, Wei-Jen Univ Elect Sci & Technol China Chengdu Sichuan Peoples R China Univ Texas Arlington Arlington TX 76019 USA

ISBN: (纸本)9781728164014

With more flexible loads and prosumers actively integrated into modern power system, competitive electricity market serves a more important role on demand side management. To facilitate residential customers flexibly participate in Demand Response (DR) programs, load aggregators (LA), as a representative of regional users, gather regional DR resources to transact with grid operators. On the basis of LAs' perspective, this paper systemically designed an uncertainty considered strategic bidding model of residential DR, aiming at improve DR reliability, meanwhile maximal the financial benefits of LAs in day-ahead market, which able to provide a guidance on reasonably estimating the DR amount of declaration, offering a wise incentive price to encourage users to accomplish the declaration, meanwhile avoiding the DR rebound peak. Moreover, to quantify the DR uncertainties, an electricity usage habits and reward sensitivities considered probabilistic model is also proposed. In addition, to optimize the continuous-action-space problem, soft actor-critic (SAC) featured deep reinforcement learning (DRL) algorithm is implemented to the decision-making process. With comparison to the traditional method and another DRL approach, SAC is proved its effectiveness in solving DR optimization problems.

关键词： Demand response Load aggregators Bidding strategy soft actor-critic algorithm power market

来源：评论

学校读者我要写书评

暂无评论

Design and implementation of a soft actor-critic controller for a robotic arm

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 151卷

作者： Kuo, Ping-Huan Huang, Chen-Ting Chang, Chen-Wen Feng, Po-Hsun Lin, Yu-Sian Natl Chung Cheng Univ Dept Mech Engn Chiayi 62102 Taiwan Natl Chung Cheng Univ Adv Inst Mfg High tech Innovat AIM HI Chiayi 62102 Taiwan Natl Cheng Kung Univ Dept Elect Engn Tainan 701401 Taiwan

In this study, reinforcement learning (RL) was employed to train and control a robotic arm with six degrees of freedom in a point tracking task. The movements of such a robotic arm are not intuitive because they are determined by the rotations of and lengths between its six joints. Given this consideration, RL can be employed for the robotic arm to learn from its mistakes and autonomously explore the effects of joint rotation of its end point position. This approach is mostly used in complex learning tasks, such as those in gaming, robotics, and automated driving. The proposed learning process was verified in a simulator, in which point tracking and obstacle avoidance tasks were performed using RL algorithms. The performance of the soft actor-critic (SAC), proximal policy optimization, advantage actor-critic, and trust region policy optimization algorithms was compared in the point tracking task, and the results indicated that the SAC algorithm outperformed the other algorithms in this task. Therefore, the SAC algorithm was used with an adequately designed reward function to conduct model training, enabling the robotic arm to achieve obstacle avoidance in an object picking task. The established model was verified to allow the robotic arm to learn from failed experiences, enable fast navigation to the target position, and facilitate rapid deployment in future application.

关键词： Robotic arm controller Reinforcement learning Obstacle avoidance Six-degree-of-freedom robotic arm soft actor-critic algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：