检索结果-内蒙古大学图书馆

approximate dynamic programming with policy-based exploration for microgrid dispatch under uncertainties

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2022年第PartB期142卷

作者： Das, Avijit Wu, Di Ni, Zhen Pacific Northwest Natl Lab Richland WA 99352 USA Atlantic Univ Boca Raton FL 33431 USA

approximate dynamic programming (ADP) is a promising approach for power system scheduling and dispatch under uncertainties. This paper presents an innovative ADP-based dispatch method for a microgrid with intermittent renewable generation, battery energy storage systems, and controllable distributed generators. The proposed ADP algorithm is based on a double-pass value iteration approach and takes advantage of the underlying properties of the microgrid dispatch problem. In the forward pass, decision variables are updated moving forward in time using an ��-greedy strategy to balance exploitation and exploration. In particular, an approximate optimization method is proposed to speed up exploitation. In addition to random exploration, a policy is designed to guide the algorithm to explore some promising solution space in a probabilistic manner. In the backward pass, the value function is updated moving backward in time using the trajectory of states, decisions, and outcomes of the sample path in the forward pass. The proposed method is evaluated through numerical experiments in both deterministic and stochastic environments. Case study results show that the proposed method demonstrates improved performance in both optimization gap and computation time in comparison to conventional methods.

关键词： approximate dynamic programming Battery energy storage systems Control policy design Microgrid Stochastic optimization

来源：评论

学校读者我要写书评

暂无评论

Anticipatory scheduling of synchromodal transport using approximate dynamic programming

引用

ANNALS OF OPERATIONS RESEARCH 2022年 1-35页

作者： Rivera, Arturo E. Perez Mes, Martijn R. K. Univ Twente High Tech Business & Entrepreneurship Dept POB 217 NL-7500 AE Enschede Netherlands

We study the problem of scheduling container transport in synchromodal networks considering stochastic demand. In synchromodal networks, the transportation modes can be selected dynamically given the actual circumstances and performance is measured over the entire network and over time. We model this problem as a Markov Decision Process and propose a heuristic solution based on approximate dynamic programming (ADP). Due to the multi-period nature of the problem, the one-step look-ahead perspective of the traditional approximate value-iteration approach can make the heuristic flounder and end in a local-optimum. To tackle this, we study the inclusion of Bayesian exploration using the Value of Perfect Information (VPI). In a series of numerical experiments, we show how VPI significantly improves a traditional ADP algorithm. Furthermore, we show how our proposed ADP-VPI combination achieves significant gains over common practice heuristics.

关键词： Synchromodal transport Intermodal transport Anticipatory scheduling approximate dynamic programming Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Lookahead approximate dynamic programming for stochastic aircraft maintenance check scheduling optimization

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2022年第3期299卷 814-833页

作者： Deng, Qichen Santos, Bruno F. Delft Univ Technol Air Transport & Operat Fac Aerosp Engn Delft Netherlands

This paper proposes a lookahead approximate dynamic programming methodology for aircraft maintenance check scheduling, considering the uncertainty of aircraft daily utilization and maintenance check elapsed time. It adopts a dynamic programming framework, using a hybrid lookahead scheduling policy. The hybrid lookahead scheduling policy makes the one-step optimal decision for heavy aircraft maintenance based on deterministic forecasts and then determines the light maintenance according to stochastic forecasts. The objective is to minimize the total wasted utilization interval between maintenance checks while reducing the need for additional maintenance slots. By achieving this goal, one is also reducing the number of maintenance checks and increasing aircraft availability while respecting airworthiness regulations. We validate the proposed methodology using the fleet maintenance data from a major European airline. The descriptive statistics of several test runs show that, when compared with the current practice, the proposed methodology potentially reduces the number of A-checks by 1.9%, the number of C-checks by 9.8%, and the number of additional slots by 78.3% over four years.

关键词： Scheduling approximate dynamic programming Lookahead policy Stochastic optimization

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for planning a ride-hailing system using autonomous fleets of electric vehicles

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2020年第3期284卷 1088-1106页

作者： Al-Kanj, Lina Nascimento, Juliana Powell, Warren B. Princeton Univ Operat Res & Financial Engn Dept Princeton NJ 08544 USA

We address a comprehensive ride-hailing system taking into account many of the decisions required to operate it in reality. The ride-hailing system is formed of a centrally managed fleet of autonomous electric vehicles which is creating a transformative new technology with significant cost savings. This problem involves a dispatch problem for assigning riders to cars, a surge pricing problem for deciding on the price per trip and a planning problem for deciding on the fleet size. We use approximate dynamic programming to develop high-quality operational dispatch strategies to determine which car is best for a particular trip, when a car should be recharged, when it should be re-positioned to a different zone which offers a higher density of trips and when it should be parked. These decisions have to be made in the presence of a highly dynamic call-in process, and assignments have to take into consideration the spatial and temporal patterns in trip demand which are captured using value functions. We prove that the value functions are monotone in the battery and time dimensions and use hierarchical aggregation to get better estimates of the value functions with a small number of observations. Then, surge pricing is discussed using an adaptive learning approach to decide on the price for each trip. Finally, we discuss the fleet size problem. (C) 2020 Elsevier B.V. All rights reserved.

关键词： approximate dynamic programming Ride-Hailing Autonomous vehicles Surge pricing Adaptive learning

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming Decompositions for Economic Dispatch in Microgrids

引用

IFAC-PapersOnLine 2022年第9期55卷 262-267页

作者： Mikhail V. Goubko Nikita I. Shushko

Three novel approximate dynamic programming algorithms based on the temporal, spatial, and spatiotemporal decomposition are proposed for the economic dispatch problem (EDP) in a distribution energy system with complex topology and many non-dispatchable renewable energy sources and energy storage systems (ESS). Computational efficiency of the proposed algorithms is compared and convergence to the optimal solution is shown in numeric experiments on the example of the two-day hourly EDP for the IEEE 33bw test network having 200+ consumers, 150+ energy storages, and 1000+ consuming devices.

关键词： approximate dynamic programming distribution network economic dispatch energy storage system energy system planning linear programming renewables’ generation spatial decomposition spatiotemporal decomposition temporal decomposition

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for the military inventory routing problem

引用

ANNALS OF OPERATIONS RESEARCH 2020年第1期288卷 391-416页

作者： McKenna, Rebekah S. Robbins, Matthew J. Lunday, Brian J. McCormack, Ian M. Air Force Inst Technol Dept Operat Sci 2950 Hobson Way Wright Patterson AFB OH 45433 USA

The United States Army can benefit from effectively utilizing cargo unmanned aerial vehicles (CUAVs) to perform resupply operations in combat environments to reduce the use of manned (ground and aerial) resupply that incurs risk to personnel. We formulate a Markov decision process (MDP) model of an inventory routing problem (IRP) with vehicle loss and direct delivery, which we label the military IRP (MILIRP). The objective of the MILIRP is to determine CUAV dispatching and routing policies for the resupply of geographically dispersed units operating in an austere, combat environment. The large size of the problem instance motivating this research renders dynamic programming algorithms inappropriate, so we utilize approximate dynamic programming (ADP) methods to attain improved policies (relative to a benchmark policy) via an approximate policy iteration algorithmic strategy utilizing least squares temporal differencing for policy evaluation. We examine a representative problem instance motivated by resupply operations experienced by the United States Army in Afghanistan both to demonstrate the applicability of our MDP model and to examine the efficacy of our proposed ADP solution methodology. A designed computational experiment enables the examination of selected problem features and algorithmic features vis-a-vis the quality of solutions attained by our ADP policies. Results indicate that a 4-crew, 8-CUAV unit is able to resupply 57% of the demand from an 800-person organization over a 3-month time horizon when using the ADP policy, a notable improvement over the 18% attained using a benchmark policy. Such results inform the development of procedures governing the design, development, and utilization of CUAV assets for the resupply of dispersed ground combat forces.

关键词： Inventory routing problem Markov decision processes approximate dynamic programming Least squares temporal differences Military

来源：评论

学校读者我要写书评

暂无评论

A kernel-based approximate dynamic programming approach: Theory and application

引用

AUTOMATICA 2024年 162卷

作者： Forootani, Ali Iervolino, Raffaele Tipaldi, Massimo Baccari, Silvio Maynooth Univ Hamilton Inst Maynooth W23F2K8 Kildare Ireland Univ Naples Federico II Dept Elect Engn & Informat Technol I-80125 Naples Italy Polytech Univ Bari Dept Elect & Informat Engn I-70126 Bari Italy Univ Campania Luigi Vanvitelli Dept Math & Phys I-81100 Caserta Italy

This article proposes a novel kernel -based dynamic programming (DP) approximation method to tackle the typical curse of dimensionality of stochastic DP problems over the finite time horizon. Such a method utilizes kernel functions in combination with Support Vector Machine (SVM) regression to determine an approximate cost function for the entire state space of the underlying Markov Decision Process (MDP), by leveraging cost function computed for selected representative states. Kernel functions are used to define the so-called kernel matrix, while the parameter vector of the given kernel -based cost function approximation is computed by moving backwards in time from the terminal condition and by applying SVM regression. This way, the difficulty of selecting a proper set of features is also tackled. The proposed method is then extended to the infinite time horizon case. To show the effectiveness of the proposed approach, the resulting Recursive Residual approximate dynamic programming (RR -ADP) algorithm is applied to the sensor scheduling design in multi -process remote state estimation problems. (c) 2024 Elsevier Ltd. All rights reserved.

关键词： dynamic programming Markov Decision Process approximate dynamic programming Kernel function Support Vector Machine Sensor scheduling

来源：评论

学校读者我要写书评

暂无评论

Flexible expansion planning of distribution system integrating multiple renewable energy sources: An approximate dynamic programming approach

引用

ENERGY 2021年 226卷 120367-120367页

作者： Sun, Qirun Wu, Zhi Gu, Wei Zhu, Tao Zhong, Lei Gao, Ting Southeast Univ Sch Elect Engn Nanjing Peoples R China Taizhou Elect Power Supply Co State Grid Zhejiang Elect Power Co Co Ltd Taizhou Peoples R China

Distribution system expansion planning is a long-term process with sequential multistage uncertain factors, so flexible planning methods are required to deal with the various potential investment risks. This paper presents an approximate dynamic programming based flexible distribution system expansion planning model, in which the long-term system load growth uncertainty and short-term power fluctuation are both considered. A flexible investment strategy based on Markov decision process is developed, in which the planning decisions are made sequentially with consideration of both the current costs and future variable costs under long-term load growth uncertainty. This problem is formulated as a large-scale multistage stochastic programming model, which is intractable due to the "curse of dimensionality". Thus, an approximate dynamic programming approach is used to decompose the original multistage optimization problem into sequential subproblems that can be easily solved. Case studies are carried out on a 24-node distribution system with three planning stages and a 54-node distribution system with five planning stages. Numerical results validate the feasibility and benefits of the proposed planning approach in obtaining a flexible expansion scheme, which can significantly reduce the investment risk and configure renewable energy equipment more reasonably. (c) 2021 Elsevier Ltd. All rights reserved.

关键词： Distribution system expansion planning Renewable energy approximate dynamic programming Markov decision process Flexible investment

来源：评论

学校读者我要写书评

暂无评论

OPEN-LOOP OPTIMAL CONTROL FOR TRACKING A REFERENCE SIGNAL WITH approximate dynamic programming

OPEN-LOOP OPTIMAL CONTROL FOR TRACKING A REFERENCE SIGNAL WI...

引用

ASME International Mechanical Engineering Congress and Exposition (IMECE)

作者： Diaz, Jorge A. Xu, Lei Sardarmehni, Tohid Univ Texas Rio Grande Valley Dept Mech Engn Edinburg TX 78539 USA Kent State Univ Dept Comp Sci Kent OH 44242 USA Calif State Univ Northridge Dept Mech Engn Northridge CA 91330 USA

ISBN: (纸本)9780791886670

dynamic programming (DP) provides a systematic, closed-loop solution for optimal control problems. However, it suffers from the curse of dimensionality in higher orders. approximate dynamic programming (ADP) methods can remedy this by finding near-optimal rather than exact optimal solutions. In summary, ADP uses function approximators, such as neural networks, to approximate optimal control solutions. ADP can then converge to the near-optimal solution using techniques such as reinforcement learning (RL). The two main challenges in using this approach are finding a proper training domain and selecting a suitable neural network architecture for precisely approximating the solutions with RL. Users select the training domain and the neural networks mostly by trial and error, which is tedious and time-consuming. This paper proposes trading the closed-loop solution provided by ADP methods for more effectively selecting the domain of training. To do so, we train a neural network using a small and moving domain around the reference signal. We asses the method's effectiveness by applying it to a widely used benchmark problem, the Van der Pol oscillator;and a real-world problem, controlling a quadrotor to track a reference trajectory. Simulation results demonstrate comparable performance to traditional methods while reducing computational requirements.

关键词： optimal control approximate dynamic programming dynamic programming neural networks

来源：评论

学校读者我要写书评

暂无评论

Solving optimal predictor-feedback control using approximate dynamic programming

引用

AUTOMATICA 2024年 170卷

作者： Wang, Hongxia Zhao, Fuyu Zhang, Zhaorong Xu, Juanjuan Li, Xun Shandong Univ Sci & Technol Sch Elect Engn & Automat Qingdao Peoples R China Shandong Univ Sch Comp Sci & Technol Qingdao Peoples R China Shandong Univ Sch Control Sci & Engn Jinan Peoples R China Hong Kong Polytech Univ Dept Appl Math Hong Kong Peoples R China

This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati-ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati-ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm. (c) 2024 Published by Elsevier Ltd.

关键词： Stochastic system Optimal control Input delay approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：