检索结果-内蒙古大学图书馆

Backward Q-learning: The combination of sarsa algorithm and Q-learning

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2013年第9期26卷 2184-2193页

作者： Wang, Yin-Hao Li, Tzuu-Hseng S. Lin, Chih-Jui Natl Cheng Kung Univ Dept Elect Engn AiRobots Lab Tainan 70101 Taiwan

Reinforcement learning (RI) has been applied to many fields and applications, but there are still some dilemmas between exploration and exploitation strategy for action selection policy. The well-known areas of reinforcement learning are the Q-learning and the sarsa algorithms, but they possess different characteristics. Generally speaking, the sarsa algorithm has faster convergence characteristics, while the Q-learning algorithm has a better final performance. However, sarsa algorithm is easily stuck in the local minimum and Q-learning needs longer time to learn. Most literatures investigated the action selection policy. Instead of studying an action selection strategy, this paper focuses on how to combine Q-learning with the sarsa algorithm, and presents a new method, called backward Q-learning, which can be implemented in the sarsa algorithm and Q-learning. The backward Q-learning algorithm directly tunes the Q-values, and then the Q-values will indirectly affect the action selection policy. Therefore, the proposed RL algorithms can enhance learning speed and improve final performance. Finally, three experimental results including cliff walk, mountain car, and cart-pole balancing control system are utilized to verify the feasibility and effectiveness of the proposed scheme. All the simulations illustrate that the backward Q-learning based RL algorithm outperforms the well-known Q-learning and the sarsa algorithm. (C) 2013 Elsevier Ltd. All rights reserved.

关键词： Backward Q-learning Q-learning Reinforcement learning sarsa algorithm

来源：评论

学校读者我要写书评

暂无评论

Residual sarsa algorithm with function approximation

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2019年第1-Sup期22卷 795-807页

作者： Fu Qiming Hu Wen Liu Quan Luo Heng Hu Lingyao Chen Jianping Suzhou Univ Sci & Technol Inst Elect & Informat Engn Suzhou 215009 Peoples R China Suzhou Univ Sci & Technol Jiangsu Key Lab Intelligent Bldg Energy Efficienc Suzhou 215009 Jiangsu Peoples R China Suzhou Univ Sci & Technol Suzhou Key Lab Mobile Networking & Appl Technol Suzhou 215009 Jiangsu Peoples R China Soochow Univ Sch Comp Sci & Technol Suzhou 215000 Jiangsu Peoples R China Jilin Univ Minist Educ Key Lab Symbol Computat & Knowledge Engn Changchun 130012 Jilin Peoples R China Collaborat Innovat Ctr Novel Software Technol & I Nanjing 210000 Jiangsu Peoples R China

In this work, we proposed an efficient algorithm named the residual sarsa algorithm with function approximation (FARS) to improve the performance of the traditional sarsa algorithm, and we use the gradient-descent method to update the function parameter vector. In the learning process, the Bellman residual method is adopted to guarantee the convergence of the algorithm, and a new rule for updating vectors of action-value functions is adopted to solve unstable and slow convergence problems. To accelerate the convergence rate of the algorithm, we introduce a new factor, named the forgotten factor, which can help improve the robustness of the algorithm's performance. Based on two classical reinforcement learning benchmark problems, the experimental results show that the FARS algorithm has better performance than other related reinforcement learning algorithms.

关键词： Reinforcement learning sarsa algorithm Function approximation Gradient descent Bellman residual

来源：评论

学校读者我要写书评

暂无评论

Urban Traffic Signal Learning Control Using sarsa algorithm Based on Adaptive RBF Network

Urban Traffic Signal Learning Control Using SARSA Algorithm ...

引用

International Conference on Measuring Technology and Mechatronics Automation

作者： Li Chun-gui Wang Meng Yang Shu-Hong Zhang Zeng-fang Guangxi Univ Technol Dept Comp Engn Liuzhou Peoples R China

ISBN: (纸本)9780769535838

Urban traffic control is very complicated, so to build a precise mathematical model for it is very difficult. In this paper, we use the sarsa reinforcement leaning algorithm to control the traffic signal, thus the decision can be made dynamically according to real-time traffic state information, and the change of environment can be adapted automatically;As the state space is too big to be stored and expressed directly, we applied radial basis function neural network to approximate the state value function. By training self-adapted non-linear processing unit, and realizing online and adaptive constructing of state space, the approximation is improved and thus the control of traffic signal at single intersections is solved. The simulation results show that the effectiveness of the new control algorithm is obviously better than traditional sliced time allocation methods.

关键词： sarsa algorithm function approximation adaptive RBF neural network traffic signal learning control

来源：评论

学校读者我要写书评

暂无评论

Research for UAV Path Planning Method Based on Guided sarsa algorithm 2

Research for UAV Path Planning Method Based on Guided Sarsa ...

引用

2nd IEEE International Conference on Software Engineering and Artificial Intelligence (SEAI) / 7th International Workshop on Pattern Recognition (IWPR)

作者： He Boming Lin Wei Mei Fuzeng Fan Huahao Wuhan Univ Technol Wuhan Peoples R China

ISBN: (数字)9781665482233

ISBN: (纸本)9781665482233

As unmanned aerial vehicles (UAV) are widely used in unknown and complex environments, their path planning capabilities face higher requirements. In many cases, UAV cannot obtain the environmental information of the target area in advance, and the reinforcement learning (RL) algorithm to solve this problem faces the problem of slow convergence speed. To solve this problem, a UAV path planning method based on guided sarsa algorithm is proposed, which defines the return function based on position information and improves the status update strategy. Simulation results show that the proposed method can realize fast path planning of UAV in static environment. Compared with Q learning and unimproved sarsa algorithm, the obtained path length is shortened by 18 steps and 2 steps. At the same time, after 2000 iterations, only the results obtained by this method have obvious convergence. The convergence speed of guided sarsa algorithm is relatively accelerated. At the same time, the trajectory tracking process of UAV is realized by MATLAB platform, which proves the practicability and feasibility of the algorithm.

关键词： UAV Path planning goal orientation reinforcement learning sarsa algorithm

来源：评论

学校读者我要写书评

暂无评论

Real-Time Network Optimization in 6G minimizing End-to-End Delay Using sarsa algorithm

Real-Time Network Optimization in 6G minimizing End-to-End D...

引用

2024 IEEE International Students' Conference on Electrical, Electronics and Computer Science, SCEECS 2024

作者： Chauhan, Tania Renjith, P.N. Vellore Institute of Technology Chennai India Vellore Institute of Technology School of Computer Science and Engineering Chennai India

ISBN: (纸本)9798350348460

As 6G networks take off, there is an increasing need for effective system optimization strategies that can decrease end-to-end (e2e) delay and improve the general efficiency of the network. The sarsa (State-Action-Reward-State-Action) algorithm, which can learn from the network ecosystem and optimize network parameters accordingly, is the foundation of the unique real-time network optimization approach that is suggested in this research. With the aid of a simulation environment that replicates the actual 6G network environment, we assess the suggested approach. Our findings show that, the sarsa algorithm may substantially decrease e2e delay and improve network performance. In general, this research offers an intriguing plan for network optimization in 6G networks and emphasizes the sarsa algorithm's potential for improving real-time network reliability. © 2024 IEEE.

关键词： 6G e2e delay Network Optimization sarsa algorithm

来源：评论

学校读者我要写书评

暂无评论

Time-Sensitive Network Simulation for In-Vehicle Ethernet Using sarsa algorithm

引用

WORLD ELECTRIC VEHICLE JOURNAL 2024年第1期15卷 21页

作者： Huang, Chen Wang, Yiqi Zhang, Yuxin Jilin Univ Coll Automot Engn Changchun 130012 Peoples R China Jiangsu Univ Inst Automot Engn Zhenjiang 212013 Peoples R China

In order to more accurately analyze the problem of time delay simulation and calculation in the time-sensitive network (TSN) of vehicular Ethernet, a TSN reservation class data delay analysis model improved based on the State-Action-Reward-State-Action (sarsa) reinforcement learning algorithm is proposed. Firstly, the TSN data queue forwarding delay model and reservation class data delay analysis intelligent body model are established, then the TSN traffic scheduling mechanism is improved by the sarsa reinforcement learning algorithm, and the improved TSN network reservation class data analysis model is established for the uncertainty of traffic scheduling in the network;finally, the fitting performance of the proposed method is verified by simulation and experimental validation. The results show that the deviation between the two is less than 5% under different BE loads, i.e., the established reservation class data delay analysis model is able to correctly fit the scheduling mechanism of the vehicle-mounted TSN network, which proves the reasonableness of the model simulation.

关键词： in-vehicle Ethernet time-sensitive networks sarsa algorithm simulation computation intelligent body modeling

来源：评论

学校读者我要写书评

暂无评论

An Online Reinforcement Learning-Based Energy Management Strategy for Microgrids With Centralized Control

引用

IEEE TRANSACTIONS ON INDUSTRY APPLICATIONS 2025年第1期61卷 1501-1510页

作者： Meng, Qinglin Hussain, Sheharyar Luo, Fengzhang Wang, Zhongguan Jin, Xiaolong State Grid Tianjin Elect Power Co Comprehens Serv Ctr Tianjin 300010 Peoples R China Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Tiangong Univ Sch Elect Engn Tianjin 300387 Peoples R China Minist Educ Key Lab Smart Grid Tianjin 300072 Peoples R China Key Lab Smart Energy & Informat Technol Tianjin Mu Tianjin 300072 Peoples R China Zhejiang Univ Inst Marine Elect & Intelligent Syst Ocean Coll Zhoushan 316000 Peoples R China

To address the issue of significant unpredictability and intermittent nature of renewable energy sources, particularly wind and solar power, this paper introduces a novel optimization model based on online reinforcement learning. Initially, an energy management optimization model is designed to achieve plan adherence and minimize energy storage (ES) operation costs, taking into account the inherent challenges of wind power-photovoltaic energy storage systems (WPESS). An online reinforcement learning framework is employed, which defines various state variables, action variables, and reward functions for the energy management optimization model. The state-action-reward-state-action (sarsa) algorithm is applied to learn the joint scheduling strategy for the microgrid system, utilizing its iterative exploration mechanisms and interaction with the environment. This strategy aims to accomplish the goals of effective power tracking and reduction of storage charging and discharging. The proposed method's effectiveness is validated using a residential community with electric vehicle (EV) charging loads as a test case. Numerical analyses demonstrate that the approach is not reliant on traditional mathematical models and adapts well to the uncertainties and complex constraints of the WPESS, maintaining a low-cost requirement while achieving computational efficiency significantly higher than that of the model predictive control (MPC) and deep Q-network (DQN) algorithm.

关键词： Reinforcement learning Renewable energy sources Microgrids Costs Monitoring Wind power generation Optimization models Grid integration optimal energy management reinforcement learning sarsa algorithm wind-solar-storage microgrid

来源：评论

学校读者我要写书评

暂无评论

Peer-to-peer Electricity Transaction Decisions of the User-side Smart Energy System Based on the sarsa Reinforcement Learning

引用

CSEE Journal of Power and Energy Systems 2022年第3期8卷 826-837页

作者： Dan Wang Bo Liu Hongjie Jia Ziyang Zhang Jingcheng Chen Deyu Huang Key Laboratory of Smart Grid of Ministry of Education Tianjin UniversityTianjin 300072China Key Laboratory of Smart Energy&Information Technology of Tianjin Municipality Tianjin 30072China State Grid Jiangsu Power Co.Ltd. Zhenjiang Power Supply CompanyZhenjiang 212000Jiangsu ProvinceChina State Grid Tianjin Electric Power Company Hebei DistrictTianjin 300010China

With the deep integration of advanced information technologies,such as artificial intelligence and traditional energy technologies,smart energy systems have been proposed as a method to provide the best solution for the coordination,balance,and control of the entire energy *** a new way of energy balance and interaction in the user side energy market,a peer-to-peer(P2P)electricity transaction can effectively promote energy sharing within the user group and improve the economic benefits of users participating in the energy *** learning(RL)is an artificial intelligence method in which agents continuously acquire relevant experience and knowledge during the interaction with the environment,automatically update their decision-making behavior;and achieve maximum *** is a suitable approach for P2P transaction decision analysis of small-scale users in the context of smart ***,this paper establishes a P2P transaction model that includes a participant model,equipment model and price ***,the transaction problem is equivalent to a Markov decision process(MDP)and each learning element model is ***,the MDP problem is solved and analyzed using the sarsa RL algorithm with average discrete ***,a case study of a community with multiple users is conducted to verify the effectiveness,economy,and security of the RL method in solving energy storage action selection and transaction decision problems of energy storage users.

关键词： Peer-to-peer electricity transaction reinforcement learning smart energy system sarsa algorithm

来源：评论

学校读者我要写书评

暂无评论

Optimal Decision-Making Method for a Plug-In Electric Taxi in Uncertain Environment

引用

IEEE ACCESS 2021年 9卷 62467-62477页

作者： You, Yang Zhu, Jisong Huang, Yichuan Jing, Zhaoxia South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China

This paper studies the optimal decision-making problem for a plug-in electric taxi (PET) in a time-varying complex environment, i.e., a passenger environment, charging station environment, traffic environment, and taxi company management system, in order to maximize PET profit in a short-term operating cycle. First, this problem is formulated as a sequential decision-making problem composed of multiple decision slots. Then, to make the model more practical, the model is divided into two parts: an external environment and an electric taxi model for refinement. The uncertainty and time-varying characteristics of four environmental aspects, including passengers, charging stations, traffic, and taxi company management systems, are analysed and modelled. The transitions between adjacent processes and the environmental feedback of each process are modelled by further subdividing both the serving process and the charging process of the PET into multiple subprocesses, including cruising, carrying passengers, driving to the charging station, queueing before charging, and connecting to the power grid for charging. There are several uncertain factors in the sequential decision-making process for the PET, which leads to difficulty in solving the problem. To address this difficulty, the model-free algorithm sarsa is chosen. Finally, the effectiveness of the proposed method is verified by simulation results.

关键词： Public transportation Decision making Electric vehicles Uncertainty Charging stations Probability distribution Power grids Plug-in electric taxi decision making uncertainty sarsa algorithm load modeling

来源：评论

学校读者我要写书评

暂无评论

A sarsa-based adaptive controller for building energy conservation

引用

JOURNAL OF COMPUTATIONAL METHODS IN SCIENCES AND ENGINEERING 2018年第2期18卷 329-338页

作者： Fu, Qiming Hu, Lingyao Wu, Hongjie Hu, Fuyuan Hu, Wen Chen, Jianping Suzhou Univ Sci & Technol Inst Elect & Informat Engn Suzhou 215009 Jiangsu Peoples R China Suzhou Univ Sci & Technol Jiangsu Key Lab Intelligent Bldg Energy Efficienc Suzhou 215009 Jiangsu Peoples R China Suzhou Univ Sci & Technol Suzhou Key Lab Mobile Networking & Appl Technol Suzhou 215009 Jiangsu Peoples R China

In the field of building equipment control, the traditional methods have some problems - instability and slow convergence. To deal with these problems, a new sarsa-based adaptive controller, SAC (sarsa-based adaptive controller) was proposed. Based on the model of the exchange mechanism about the building energy consumption, the proposed method with sarsa algorithm models the exchange mechanism of the building energy consumption, and tries to find the best control policy, which can decrease the energy consumption without losing the performance of good comfort of the building occupants. Compared with the PID method, the proposed SAC has better convergence performance and robustness.

关键词： Energy conservation Reinforcement learning sarsa algorithm adaptive controller

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：