检索结果-内蒙古大学图书馆

Offline-Online approximate dynamic programming for dynamic Vehicle Routing with Stochastic Requests

TRANSPORTATION SCIENCE 2019年第1期53卷 185-202页

作者： Ulmer, Marlin W. Goodson, Justin C. Mattfeld, Dirk C. Hennig, Marco Tech Univ Carolo Wilhelmina Braunschweig D-38106 Braunschweig Germany St Louis Univ Richard A Chaifetz Sch Business St Louis MO 63103 USA

Although increasing amounts of transaction data make it possible to characterize uncertainties surrounding customer service requests, few methods integrate predictive tools with prescriptive optimization procedures to meet growing demand for small-volume urban transport services. We incorporate temporal and spatial anticipation of service requests into approximate dynamic programming (ADP) procedures to yield dynamic routing policies for the single-vehicle routing problem with stochastic service requests, an important problem in city-based logistics. We contribute to the routing literature as well as to the field of ADP. We combine offline value function approximation (VFA) with online rollout algorithms resulting in a high-quality, computationally tractable policy. Our offline-online policy enhances the anticipation of the VFA policy, yielding spatial and temporal anticipation of requests and routing developments. Our combination of VFA with rollout algorithms demonstrates the potential benefit of using offline and online methods in tandem as a hybrid ADP procedure, making possible higher-quality policies with reduced computational requirements for real-time decision making. Finally, we identify a policy improvement guarantee applicable to VFA-based rollout algorithms, showing that base policies composed of deterministic decision rules lead to rollout policies with performance at least as strong as that of their base policy.

关键词： dynamic vehicle routing stochastic customer requests approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

ONLINE CAPACITY PLANNING FOR REHABILITATION TREATMENTS: AN approximate dynamic programming APPROACH

引用

PROBABILITY IN THE ENGINEERING AND INFORMATIONAL SCIENCES 2020年第3期34卷 381-405页

作者： Bikker, Ingeborg A. Mes, Martijn R. K. Saure, Antoine Boucherie, Richard J. Univ Twente Ctr Healthcare Operat Improvement & Res CHOIR Drienerlolaan 5 NL-7500 AE Enschede Netherlands Sint Maartensklin Dept Healthcare Logist Hengstdal 3 NL-6574 NA Nijmegen Netherlands Univ Twente Dept Appl Math Stochast Operat Res Drienerlolaan 5 NL-7500 AE Enschede Netherlands Univ Twente Dept Ind Engn & Business Informat Syst IEBIS Drienerlolaan 5 NL-7500 AE Enschede Netherlands Univ Ottawa Telfer Sch Management 55 Laurier Ave East Ottawa ON K1N 6N5 Canada

We study an online capacity planning problem in which arriving patients require a series of appointments at several departments, within a certain access time target. This research is motivated by a study of rehabilitation planning practices at the Sint Maartenskliniek hospital (the Netherlands). In practice, the prescribed treatments and activities are typically booked starting in the first available week, leaving no space for urgent patients who require a series of appointments at a short notice. This leads to the rescheduling of appointments or long access times for urgent patients, which has a negative effect on the quality of care and on patient satisfaction. We propose an approach for allocating capacity to patients at the moment of their arrival, in such a way that the total number of requests booked within their corresponding access time targets is maximized. The model considers online decision making regarding multi-priority, multi-appointment, and multi-resource capacity allocation. We formulate this problem as a Markov decision process (MDP) that takes into account the current patient schedule, and future arrivals. We develop an approximate dynamic programming (ADP) algorithm to obtain approximate optimal capacity allocation policies. We provide insights into the characteristics of the optimal policies and evaluate the performance of the resulting policies using simulation.

关键词： approximate dynamic programming healthcare logistics online capacity planning operations research Markov decision process rehabilitation treatment planning simulation

来源：评论

学校读者我要写书评

暂无评论

Stochastic economic dispatch of power system with multiple wind farms and pumped-storage hydro stations using approximate dynamic programming

引用

IET RENEWABLE POWER GENERATION 2020年第13期14卷 2507-2516页

作者： Lin, Shunjiang Fan, Guansheng Jian, Ganyang Liu, Mingbo South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China China Southern Power Grid Elect Power Res Inst Guangzhou 510663 Peoples R China

The stochastic economic dispatch problem of power system with multiple wind farms and pumped-storage hydro stations is formulated as a specific stochastic dynamic programming (DP) model, i.e. stochastic storage model, it is impossible to obtain an accurate solution due to the curse of dimensionality. Based on the approximate DP (ADP) method, the stochastic storage model can be transformed into a series of mixed-integer linear programming (MILP) models by describing the approximate value functions (AVFs) as convex piecewise linear functions in post-decision states. The AVFs are first initialised using the results of the deterministic model under a forecast scenario of wind farm output and then trained by scanning stochastic sampling scenarios consecutively with the successive projective approximation routine algorithm. To obtain a near-optimal day-ahead dispatch scheme, the forecast scenario is substituted into the MILP models expressed by the trained AVFs and is solved forward through each time interval. The network constraints are incorporated by the while-loop detection of critical lines. Test results on an actual provincial power system and the modified IEEE 39-bus system, including the comparison among the ADP, DP, scenario-based and chance-constrained programming methods, demonstrate the feasibility and efficiency of the proposed model and algorithm.

关键词： approximation theory stochastic processes dynamic programming power generation economics wind power plants power generation dispatch pumped-storage power stations integer programming stochastic programming power system security optimisation linear programming multiple wind farms pumped-storage hydro stations approximate dynamic programming stochastic economic dispatch problem specific stochastic dynamic programming model stochastic storage model approximate DP method mixed-integer linear programming models approximate value functions AVFs convex piecewise linear functions deterministic model forecast scenario wind farm output stochastic sampling scenarios successive projective approximation routine algorithm near-optimal day-ahead dispatch scheme MILP models actual provincial power system modified IEEE 39-bus system

来源：评论

学校读者我要写书评

暂无评论

Control of a Buck DC/DC Converter Using approximate dynamic programming and Artificial Neural Networks

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS 2021年第4期68卷 1760-1768页

作者： Dong, Weizhen Li, Shuhui Fu, Xingang Li, Zhongwen Fairbank, Michael Gao, Yixiang Univ Alabama Dept Elect & Comp Engn Tuscaloosa AL 35487 USA Texas A&M Univ Dept Elect Engn & Comp Sci Kingsville TX 78363 USA Zhengzhou Univ Sch Elect Engn Zhengzhou 450001 Peoples R China Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England

This paper proposes a novel artificial neural network (ANN) based control method for a dc/dc buck converter. The ANN is trained to implement optimal control based on approximate dynamic programming (ADP). Special characteristics of the proposed ANN control include: 1) The inputs to the ANN contain error signals and integrals of the error signals, enabling the ANN to have PI control ability;2) The ANN receives voltage feedback signals from the dc/dc converter, making the combined system equivalent to a recurrent neural network;3) The ANN is trained to minimize a cost function over a long time horizon, making the ANN have a stronger predictive control ability than a conventional predictive controller;4) The ANN is trained offline, preventing the instability of the network caused by weight adjustments of an on-line training algorithm. The ANN performance is evaluated through simulation and hardware experiments and compared with conventional control methods, which shows that the ANN controller has a strong ability to track rapidly changing reference commands, maintain stable output voltage for a variable load, and manage maximum duty-ratio and current constraints properly.

关键词： Voltage control Buck converters Control systems Switching frequency Transfer functions Inductors Switches dc dc buck converter artificial neural network approximate dynamic programming optimal control

来源：评论

学校读者我要写书评

暂无评论

An approximate dynamic programming approach for the vehicle routing problem with stochastic demands

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2009年第2期196卷 509-515页

作者： Novoa, Clara Storer, Robert SW Texas State Univ Ingram Sch Engn Ind Engn Program San Marcos TX 78666 USA Lehigh Univ Dept Ind & Syst Engn Bethlehem PA 18015 USA

This paper examines approximate dynamic programming algorithms for the single-vehicle routing problem with stochastic demands from a dynamic or reoptimization perspective. The methods extend the rollout algorithm by implementing different base sequences (i.e. a priori solutions), look-ahead policies, and pruning schemes. The paper also considers computing the cost-to-go with Monte Carlo simulation in addition to direct approaches. The best new method found is a two-step lookahead rollout started with a stochastic base sequence. The routing cost is about 4.8% less than the one-step rollout algorithm started with a deterministic sequence. Results also show that Monte Carlo cost-to-go estimation reduces computation time 65% in large instances with little or no loss in solution quality. Moreover, the paper compares results to the perfect information case from solving exact a posteriori solutions for sampled vehicle routing problems. The confidence interval for the overall mean difference is (3.56%, 4.11%). (C) 2008 Elsevier B.V. All rights reserved.

关键词： Transportation Stochastic vehicle routing approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Novel time-space network flow formulation and approximate dynamic programming approach for the crane scheduling in a coil warehouse

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2017年第2期262卷 424-437页

作者： Yuan, Yuan Tang, Lixin Northeastern Univ Inst Ind & Syst Engn Shenyang Peoples R China

This article proposes an efficient event-based time-space network flow model with side constraints for the crane scheduling problem in a coil warehouse where the crane should carry out a set of coil storage, retrieval and shuffling requests, and determine the sequence of handling these requests as well as the positions to which the coils are moved. The model is formulated based on a graph such that each node represents a location in the warehouse at the end of a specific scheduling stage, and each edge indicates a crane's move between two locations in a stage. Variables reduction strategies are presented to accelerate solving the model. In order to solve large-sized instances of the problem, an exact dynamic programming approach based on optimal assignments between coils and positions in a bipartite network with cuts is designed by exploiting the problem structure. Then an approximate dynamic programming (ADP) approach is developed, in which an affine value function approximation is defined as the estimation of crane's traveling time for handling each coil, and updated via iterations by collecting information from the solutions of separate subproblems. Computational results show that the proposed model is tighter and can be solved much more quickly than a traditional model for a reduced crane scheduling problem in the literature and the standard time-space network flow model. Besides, the proposed algorithm can obtain high quality solutions for large-sized instances in a few minutes and is more efficient in solving the problem than a commercial software package. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Scheduling Logistics Coil warehouse Time-space network flow formulation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Horizontal combinations of online and offline approximate dynamic programming for stochastic dynamic vehicle routing

引用

CENTRAL EUROPEAN JOURNAL OF OPERATIONS RESEARCH 2020年第1期28卷 279-308页

作者： Ulmer, Marlin W. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany

Stochastic and dynamic vehicle routing problems gain increasing attention in the research community. In these problems, routing plans are dynamically updated based on realizations of stochastic information. Due to the complexity of the corresponding Markov decision processes (MDPs), the calculation of optimal policies for these problems is usually not possible and researchers draw on heuristical methods of approximate dynamic programming (ADP). These methods use simulation to approximate the value of a state and decision in the MDP. The simulations are either conducted offline or online. Offline methods such as value function approximations (VFAs) generally neglect the full detail of the state space due to aggregation. Online methods such as rollout algorithms (RAs) are often not able to capture decision and transition space sufficiently due to runtime limitations. In this paper, we alleviate this tradeoff by combining two methods of ADP, an online RA and an offline VFA in two ways. In addition to the integration of the VFA as a base policy into the online RA to strengthen the RA's simulations, we also limit the RA's simulation horizon, estimating the remaining reward-to-go again via the VFA. For two stochastic dynamic routing problems from the literature, we show how this combination outperforms state-of-the-art solutions while simultaneously reducing the required time for online calculations.

关键词： approximate dynamic programming Rollout algorithm Value function approximation dynamic vehicle routing Stochastic requests Multi-period

来源：评论

学校读者我要写书评

暂无评论

Sub-optimal switching in anti-lock brake systems using approximate dynamic programming

引用

IET CONTROL THEORY AND APPLICATIONS 2019年第9期13卷 1413-1424页

作者： Sardarmehni, Tohid Heydari, Ali Texas A&M Univ Engn Technol & Ind Distribut College Stn TX 77843 USA Southern Methodist Univ Mech Engn Dallas TX 75205 USA

Optimal scheduling in an anti-lock brake system of ground vehicles is performed through approximate dynamic programming for reducing the stopping distance in severe braking. The proposed optimal scheduler explicitly incorporates the hybrid nature of the anti-lock brake system and provides a feedback solution with a negligible computational burden in control calculation. To this goal, an iterative scheme, called the value iteration algorithm, is used to derive the infinite horizon solution to the underlying Hamilton-Jacobi-Bellman equation. Performance of the proposed method in control of the brake system is illustrated using both linear-in-parameter neural networks and multi-layer perceptrons. Simulation results demonstrate potentials of the method.

关键词： optimal control dynamic programming brakes scheduling feedback braking iterative methods multilayer perceptrons infinite horizon road traffic control vehicle dynamics approximate dynamic programming optimal scheduling anti-lock brake system sub-optimal switching ground vehicles feedback solution value iteration algorithm infinite horizon solution Hamilton-Jacobi-Bellman equation linear-in-parameter neural networks multilayer perceptrons

来源：评论

学校读者我要写书评

暂无评论

Risk-Averse approximate dynamic programming with Quantile-Based Risk Measures

引用

MATHEMATICS OF OPERATIONS RESEARCH 2018年第2期43卷 554-579页

作者： Jiang, Daniel R. Powell, Warren B. Univ Pittsburgh Dept Ind Engn Pittsburgh PA 15261 USA Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08540 USA

In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs;we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measures that includes the popular value at risk (VaR) and the conditional value at risk (CVaR). Although there is considerable theoretical development of risk-averse MDPs in the literature, the computational challenges have not been explored as thoroughly. We propose data-driven and simulation-based approximate dynamic programming (ADP) algorithms to solve the risk-averse sequential decision problem. We address the issue of inefficient sampling for risk applications in simulated settings and present a procedure, based on importance sampling, to direct samples toward the "risky region" as the ADP algorithm progresses. Finally, we show numerical results of our algorithms in the context of an application involving risk-averse bidding for energy storage.

关键词： approximate dynamic programming dynamic risk measures energy trading reinforcement learning Q-learning

来源：评论

学校读者我要写书评

暂无评论

Gaussian Process approximate dynamic programming for Energy-Optimal Supervisory Control of Parallel Hybrid Electric Vehicles

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2022年第8期71卷 8367-8380页

作者： Bae, Jin Woo Kim, Kwang-Ki K. Texas A&M Univ Dept Comp Sci & Engn College Stn TX 77843 USA Inha Univ Dept Elect & Comp Engn Incheon 22212 South Korea

We propose an energy-efficient supervisory control method for the power management of parallel hybrid electric vehicles (HEVs) to improve the fuel economy and reduce exhaust gas emissions. Plug-in HEVs ((P)HEVs) have multiple power sources (e.g., an engine and motor) that should be cooperatively operated to meet the required instantaneous traction power for the desired vehicle speed while satisfying their physical limits. Because the efficiencies of the engine and motor vary with different operating speeds and torques, the main issue of energy-efficient power management is to allocate the power demand among the power sources by achieving maximum power conversion efficiencies and satisfy the operating limits. For an efficient power allocation, an optimal control problem is formulated, and a global solution is found through deterministic dynamic programming (DP). Owing to the curse of dimensionality and uncertainties in real driving, DP solutions are not directly applicable in real time. To resolve the limitations of DP, we employ a non-parametric Bayesian function approximation technique using a Gaussian process (GP). The offline DP solutions obtained from a set of real vehicle driving test data were used to learn a state-dependent probabilistic value function through Gaussian process regression. For online implementations, a receding horizon control scheme was applied for the feedback control of the power management. In comparison with the existing charge sustaining strategy and charge depleting and charge sustaining mixed controllers, we recorded fuel efficiency improvements of over 4.8% and 7.3%, respectively, in a mixed urban-suburban route.

关键词： Batteries Vehicle dynamics Mathematical models Hybrid electric vehicles Energy management Optimal control Engines approximate dynamic programming energy management gaussian process regression optimal control Parallel hybrid electric vehicles supervisory control value function approximation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：