检索结果-内蒙古大学图书馆

Optimal Charging Strategy for EV Charging Stations by Two-Stage approximate dynamic programming

IFAC Proceedings Volumes 2013年第5期46卷 423-430页

作者： Lei Zhang Yaoyu Li Department of Electrical Engineering University of Texas at Dallas Richardson TX 75080 USA Department of Mechanical Engineering University of Texas at Dallas Richardson TX 75080 USA (Tel: 972-883-4698

This study targets the charging scenario for multiple chargeable vehicles at parking lots or charging stations where most vehicles have extended parking time, e.g. at the commercial buildings. All the vehicles are desired to be charged to full prior to the departure time specified, under constrained total charging capacity. A two-stage approximate dynamic programming (TSADP) framework is proposed to determine the optimal charging strategy, utilizing the predicted short-term future information and long-term estimation from historical data. The algorithm is separated into two stages, the optimization stage and approximation stage. The optimization stage is to find the optimal charging strategy for the short-term ahead time horizon. The approximation stage is to approximate the long-term time horizon charging cost, by the way of approximate dynamic programming (ADP). The whole algorithm works as a partial rollout algorithm and partial approximate dynamic programming. The TSADP framework can be integrated into stochastic programming framework, when the short-term information is uncertain with probabilities. The simulation results show that the proposed method can significantly decrease the energy cost.

关键词： Electric Vehicle Charging Control approximate dynamic programming Stochastic programming Prediction

来源：评论

学校读者我要写书评

暂无评论

Accelerating value function approximations for dynamic dial-a-ride problems via dimensionality reductions

引用

COMPUTERS & OPERATIONS RESEARCH 2024年 167卷

作者： Heitmann, R. -Julius O. Soeffker, Ninja Klawonn, Frank Ulmer, Marlin W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany Univ Wien Vienna Austria Otto von Guericke Univ Magdeburg Germany

As the success of ride -sharing mobility service providers shows, customer demand for shared mobility services is increasing. The availability of mobile devices enables the constant accessibility of mobility apps and the immediate placement of transport requests. To provide such a dynamic dial -a -ride service, an effective control of the fleet is necessary. One promising solution approach is the value function approximation (VFA), which on the one hand convinces through good performance, but on the other hand also stands out through fast response times for a request. Training a VFA can be a challenging task since, among other things, the dimensionality of the state space plays a decisive role. If many variables to describe a state are used, a high amount of information can produce good performance after completion of the learning process. If the state space is too high -dimensional, there is also a risk that the method will not be able to find a reasonable solution. In contrast, if the number of variables is reduced, the learning speed can be accelerated, but the eventual performance may suffer from the associated loss of information. Furthermore, not all variables are equally relevant, as they contain different amounts of information. This paper presents a hybrid strategy, temporarily lowering the dimensionality of the problem using dimension reduction methods and subsequently increasing it by mapping the lower -dimensional state representations back onto a high -dimensional state space in order to exploit the advantages of both space dimensionalities. VFA in itself results in competitive performance for the dynamic dial -a -ride problem with shared rides. The proposed hybrid state representation can outperform the reference state representations by 3%, which corresponds to a meaningful acceleration in VFA learning speed.

关键词： Ride-sharing Dial-a-ride dynamic vehicle routing approximate dynamic programming Reinforcement learning Dimensionality reduction

来源：评论

学校读者我要写书评

暂无评论

Output feedback fault-tolerant Q-learning for discrete-time linear systems with actuator faults

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2024年 138卷

作者： Rafiee, Sajad Kankashvar, Mohammadrasoul Bolandi, Hossein Iran Univ Sci & Technol Dept Control Syst Engn Tehran *** Iran

This paper presents an innovative output feedback fault-tolerant Q-learning algorithm that can be implemented online without relying on explicit system models or fault details. In the face of actuator faults, finding optimal Fault-Tolerant Control (FTC) solutions that can stabilize the faulty system poses significant challenges. The proposed approach is implemented online without the need for system dynamics and actuator fault information. Furthermore, it operates without relying on full state measurements, utilizing only the input-output data of the faulty system. An innovative representation of the output feedback Fault-Tolerant Q-function (FTQF) is established using input-output data. Subsequently, a model-free optimal output feedback FTC policy is obtained from the developed FTQF. Then, a fault-tolerant Q-learning algorithm is formulated to iteratively acquire the optimal FTC policy in real-time, eliminating the necessity for system dynamics and actuator fault information. The proposed algorithm exhibits immunity to excitation noise bias, even without considering a discounting factor. Furthermore, the proposed Q-learning approach is proven to be effective in stabilizing faulty closed-loop system. Finally, the proposed algorithm's efficiency is validated through numerical simulations of F-16 autopilot aircraft dynamics.

关键词： Reinforcement learning Fault-tolerant control Linear quadratic regulator Output feedback Optimal control approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems

引用

SENSORS 2024年第17期24卷 5462页

作者： Li, Shu Ren, Tao Ding, Liang Liu, Lei Liaoning Univ Technol Key Lab Intelligent Control Theory & Applicat Liao Jinzhou 121001 Peoples R China Harbin Inst Technol State Key Lab Robot & Syst Harbin 150001 Peoples R China

For nonlinear systems with uncertain state time delays, an adaptive neural optimal tracking control method based on finite time is designed. With the help of the appropriate LKFs, the time-delay problem is handled. A novel nonquadratic Hamilton-Jacobi-Bellman (HJB) function is defined, where finite time is selected as the upper limit of integration. This function contains information on the state time delay, while also maintaining the basic information. To meet specific requirements, the integral reinforcement learning method is employed to solve the ideal HJB function. Then, a tracking controller is designed to ensure finite-time convergence and optimization of the controlled system. This involves the evaluation and execution of gradient descent updates of neural network weights based on a reinforcement learning architecture. The semi-global practical finite-time stability of the controlled system and the finite-time convergence of the tracking error are guaranteed.

关键词： approximate dynamic programming wheeled mobile robotics tracking control state time delay finite time

来源：评论

学校读者我要写书评

暂无评论

Mean Field Game-based Decentralized Optimal Charging Control for large-scale of electric Vehicles 6

Mean Field Game-based Decentralized Optimal Charging Control...

引用

6th IFAC Conference on Intelligent Control and Automation Sciences (ICONS)

作者： Zhou, Zejian Xu, Hao Univ New Haven Dept Elect & Comp Engn & Comp Sci West Haven CT 06516 USA Univ Nevada Dept Elect & Biomed Engn Reno NV 89557 USA

In this paper, the online decentralized optimal charging control problem for large-scale electric vehicles is investigated. Massive concurrent charging, which often happens for the electric buses in a smart city, is devastating to the power grid's supply capacity due to the instant high load demand. Traditional strategies rely on a centralized dispatch center which suffers from the rising computational complexity and the data communication burden when the bus number is huge. In the novel design, a decentralized optimal charging control strategy is developed to tackle these challenges. By engaging the emerging Mean Field Games (MFG) theory with a novel Actor-Critic-Mass (ACM) reinforcement learning scheme, a probability density function (PDF) of the massive state of charge (SOC) is used to encode all electric vehicles' charging demand information into a fix-dimension function. Moreover, the optimal charging control and the PDF of massive SOC can be solved through the coupled Fokker-Planck-Kolmogorov (FPK) and Hamilton-Jacobi-Bellman (HJB) equations in a decentralized manner. Eventually, the effectiveness of the proposed scheme is validated through a series of numerical simulations. Copyright (c) 2022 The Authors. This is an open access article under the CC BY-NC-ND license (https://***/licenses/by-nc-nd/4.0/)

关键词： Mean Field Games reinforcement learning approximate dynamic programming multi-agent systems reinforcement learning control

来源：评论

学校读者我要写书评

暂无评论

Addressing state space multicollinearity in solving an ozone pollution dynamic control problem

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2021年第2期289卷 683-695页

作者： Ariyajunya, Bancha Chen, Ying Chen, Victoria C. P. Kim, Seoung Bum Rosenberger, Jay Burapha Univ Fac Engn Chon Buri Thailand Harbin Inst Technol Dept Management Sci & Engn Harbin Peoples R China Univ Texas Arlington Dept Ind Mfg & Syst Engn Arlington TX 76019 USA Korea Univ Sch Ind Management Engn Seoul South Korea

High ground-level ozone concentrations constitute a serious air quality problem in many metropolitan regions. In this paper, we study a stochastic dynamic programming (SDP) formulation of the Atlanta metropolitan ozone pollution problem that seeks to reduce ozone via reductions of nitrogen oxides. The initial SDP formulation involves a 524-dimensional continuous state space, including ozone concentrations that are highly correlated. In prior work, a design and analysis of computer experiments (DACE) based approximate dynamic programming (ADP) solution method was able to conduct dimensionality reduction and value function approximation to enable a computationally-tractable numerical solution. However, this prior work did not address state space multicollinearity. In statistical modeling, high multicollinearity is well-known to adversely affect the generalizability of the constructed model. This issue is relevant whenever an empirical model is trained on data, but is largely ignored in the ADP literature. We propose approaches for addressing the multicollinearity in the Atlanta case study and demonstrate that if high multicollinearity is ignored, the resulting empirical models provide misleading information within the ADP algorithm. Because many SDP applications involve multicollinear continuous state spaces, the lessons learned in our research can guide the development of ADP approaches for a wide variety of SDP problems. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Ozone pollution Computer experiments Multicollinearity Statistical modeling approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Discrete-Time Non-Zero-Sum Games With Completely Unknown dynamics

引用

IEEE TRANSACTIONS ON CYBERNETICS 2021年第6期51卷 2929-2943页

作者： Song, Ruizhuo Wei, Qinglai Zhang, Huaguang Lewis, Frank L. Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Peoples R China Univ Texas Arlington UTA Res Inst Ft Worth TX 76118 USA

In this article, off-policy reinforcement learning (RL) algorithm is established to solve the discrete-time N-player nonzero-sum (NZS) games with completely unknown dynamics. The N-coupled generalized algebraic Riccati equations (GARE) are derived, and then policy iteration (PI) algorithm is used to obtain the N-tuple of iterative control and iterative value function. As the system dynamics is necessary in PI algorithm, off-policy RL method is developed for discrete-time N-player NZS games. The off-policy N-coupled Hamilton-Jacobi (HJ) equation is derived based on quadratic value functions. According to the Kronecker product, the N-coupled HJ equation is decomposed into unknown parameter part and the system operation data part, which makes the N-coupled HJ equation solved independent of system dynamics. The least square is used to calculate the iterative value function and N-tuple of iterative control. The existence of Nash equilibrium is proved. The result of the proposed method for discrete-time unknown dynamics NZS games is indicated by the simulation examples.

关键词： Adaptive critic designs adaptive dynamic programming approximate dynamic programming discrete-time nonzero-sum (NZS) off-policy reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Online Allocation and Pricing: Constant Regret via Bellman Inequalities

引用

OPERATIONS RESEARCH 2021年第3期69卷 821-840页

作者： Vera, Alberto Banerjee, Siddhartha Gurvich, Itai Cornell Univ Sch Operat Res & Informat Engn Ithaca NY 14853 USA Northwestern Univ Kellogg Sch Management Evanston IL 60208 USA

We develop a framework for designing simple and efficient policies for a family of online allocation and pricing problems that includes online packing, budget-constrained probing, dynamic pricing, and online contextual bandits with knapsacks. In each case, we evaluate the performance of our policies in terms of their regret (i.e., additive gap) relative to an offline controller that is endowed with more information than the online controller. Our framework is based on Bellman inequalities, which decompose the loss of an algorithm into two distinct sources of error: (1) arising from computational tractability issues, and (2) arising from estimation/prediction of random trajectories. Balancing these errors guides the choice of benchmarks, and leads to policies that are both tractable and have strong performance guarantees. In particular, in all our examples, we demonstrate constant-regret policies that only require resolving a linear program in each period, followed by a simple greedy action-selection rule;thus, our policies are practical as well as provably near optimal.

关键词： stochastic optimization approximate dynamic programming online resource allocation dynamic pricing online packing network revenue management

来源：评论

学校读者我要写书评

暂无评论

Real-time dispatch management of shared autonomous vehicles with on-demand and pre-booked requests

引用

TRANSPORTATION RESEARCH PART A-POLICY AND PRACTICE 2024年 181卷

作者： Chen, Yao Liu, Yang Baia, Yun Mao, Baohua Beijing Jiaotong Univ Key Lab Transport Ind Big Data Applicat Technol Co Beijing Peoples R China Natl Univ Singapore Dept Civil & Environm Engn Singapore Singapore Natl Univ Singapore Dept Ind Syst Engn & Management Singapore Singapore

Autonomous vehicle technology is poised to revolutionize shared vehicle systems, offering the potential for increased efficiency and convenience. To better devise management strategies for shared autonomous vehicles, this paper addresses a real-time dispatch problem with hybrid requests, where on -demand (immediate) and pre -booked (reserved) trip requests coexist. The coexistence of these two types of request behaviors introduces considerable complexity to realtime dispatch due to the uncertainty in trip demand. We design an approximate dynamic programming (ADP) approach for making vehicle-trip assignments and vehicle relocation decisions. We first formulate the real-time vehicle dispatch problem as a dynamic program and decompose it into time -staged subproblems. To effectively handle the high -dimensional state space, we replace the value functions with tractable approximations and propose a piecewiselinear functional approximation method that captures the spatiotemporal value of vehicles. To calibrate the parameters in the approximations, we propose DualT and DualNext algorithms to provide precise dual information, thereby enhancing the accuracy of our approach. Furthermore, we propose a lookahead strategy that incorporates pre -booked request information into the ADP approach for improving real-time decision -making. We validate the effectiveness of the ADP approach through numerical experiments conducted using taxi data from Brooklyn, New York. The ADP approach outperforms benchmark policies in solution quality while maintaining computational efficiency, and the incorporation of the lookahead strategy significantly enhances the performance of the ADP approach, yielding substantial improvements. Numerical results demonstrate that integrating pre -booked requests into vehicle dispatch management can greatly enhance the system efficiency.

关键词： Shared autonomous vehicles dynamic fleet management approximate dynamic programming Vehicle dispatch Reservation Lookahead

来源：评论

学校读者我要写书评

暂无评论

An approximated dynamic programming model for the supply vessel fleet sizing problem

引用

COMPUTERS & OPERATIONS RESEARCH 2024年 163卷

作者： Ribeiro, Leonardo Souza Vistex GmbH Altrottstr 31 D-69190 Walldorf Germany Petrobras SA Ave Chile 65 Rio De Janeiro Brazil Sickingerweg 20 D-69190 Walldorf Germany

The offshore oil and gas exploration and production activities heavily rely on a fleet of supply vessels, which play a crucial role in regularly providing necessary provisions from an onshore supply depot to the offshore installations. For Petrobras, the decision-making process concerning the number of supply vessels to be hired for the upcoming two years holds significant economic implications, given that the daily rates for a single supply vessel can soar to tens of thousands of *** this study, we address the challenges of this heterogeneous fleet sizing problem by presenting an innovative approximated dynamic programming (ADP) model. The model takes into account the uncertainties associated with sea conditions and future demand, making it well-suited for real-world applications. To tackle this problem, we explored two types of approximate value functions: a piecewise linear concave value function and a neural network value function. Both methods underwent rigorous testing and demonstrated their capability to effectively address real-world problems. Consequently, our proposed method exhibits promising potential for practical implementation, allowing companies like Petrobras to accurately assess and determine the ideal fleet composition. By introducing this ADP model, we aim to provide valuable insights and solutions that enhance decision-making processes related to supply vessel fleet sizing in offshore operations. Through considering various uncertainties and complexities, our approach offers a robust and reliable framework to support critical economic decisions in this dynamic industry.

关键词： Oil and gas Logistics Fleet size approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：