检索结果-内蒙古大学图书馆

Policy Iteration Based approximate dynamic programming Toward Autonomous Driving in Constrained dynamic Environment

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2023年第5期24卷 5003-5013页

作者： Lin, Ziyu Ma, Jun Duan, Jingliang Li, Shengbo Eben Ma, Haitong Cheng, Bo Lee, Tong Heng Tsinghua Univ Sch Vehicle & Mobil Beijing 100084 Peoples R China Hong Kong Univ Sci & Technol Guangzhou Robot & Autonomous Syst Thrust Guangzhou Peoples R China Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong Peoples R China Univ Sci & Technol Beijing Sch Mech Engn Beijing 100083 Peoples R China Natl Univ Singapore Dept Elect & Comp Engn Singapore 117583 Singapore

In the area of autonomous driving, it typically brings great difficulty in solving the motion planning problem since the vehicle model is nonlinear and the driving scenarios are complex. Particularly, most of the existing methods cannot be generalized to dynamically changing scenarios with varying surrounding vehicles. To address this problem, this development here investigates the framework of integrated decision and control. As part of the modules, static path planning determines the reference candidates ahead, and then the optimal path-tracking controller realizes the specific autonomous driving task. An innovative and effective constrained finite-horizon approximate dynamic programming (ADP) algorithm is herein presented to generate the desired control policy for effective path tracking. With the generalized policy neural network that maps from the state to the control input, the proposed algorithm preserves the high effectiveness for the motion planning problem towards changing driving environments with varying surrounding vehicles. Moreover, the algorithm attains the noteworthy advantage of alleviating the typically heavy computational loads with the mode of offline training and online execution. As a result of the utilization of multi-layer neural networks in conjunction with the actor-critic framework, the constrained ADP method is capable of handling complex and multidimensional scenarios. Finally, various simulations have been carried out to show that the constrained ADP algorithm is effective.

关键词： Planning Autonomous vehicles Vehicle dynamics Task analysis Heuristic algorithms Approximation algorithms Roads Autonomous driving approximate dynamic programming motion planning constrained optimization reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Offline-Online approximate dynamic programming for dynamic Vehicle Routing with Stochastic Requests

引用

TRANSPORTATION SCIENCE 2019年第1期53卷 185-202页

作者： Ulmer, Marlin W. Goodson, Justin C. Mattfeld, Dirk C. Hennig, Marco Tech Univ Carolo Wilhelmina Braunschweig D-38106 Braunschweig Germany St Louis Univ Richard A Chaifetz Sch Business St Louis MO 63103 USA

Although increasing amounts of transaction data make it possible to characterize uncertainties surrounding customer service requests, few methods integrate predictive tools with prescriptive optimization procedures to meet growing demand for small-volume urban transport services. We incorporate temporal and spatial anticipation of service requests into approximate dynamic programming (ADP) procedures to yield dynamic routing policies for the single-vehicle routing problem with stochastic service requests, an important problem in city-based logistics. We contribute to the routing literature as well as to the field of ADP. We combine offline value function approximation (VFA) with online rollout algorithms resulting in a high-quality, computationally tractable policy. Our offline-online policy enhances the anticipation of the VFA policy, yielding spatial and temporal anticipation of requests and routing developments. Our combination of VFA with rollout algorithms demonstrates the potential benefit of using offline and online methods in tandem as a hybrid ADP procedure, making possible higher-quality policies with reduced computational requirements for real-time decision making. Finally, we identify a policy improvement guarantee applicable to VFA-based rollout algorithms, showing that base policies composed of deterministic decision rules lead to rollout policies with performance at least as strong as that of their base policy.

关键词： dynamic vehicle routing stochastic customer requests approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

ONLINE CAPACITY PLANNING FOR REHABILITATION TREATMENTS: AN approximate dynamic programming APPROACH

引用

PROBABILITY IN THE ENGINEERING AND INFORMATIONAL SCIENCES 2020年第3期34卷 381-405页

作者： Bikker, Ingeborg A. Mes, Martijn R. K. Saure, Antoine Boucherie, Richard J. Univ Twente Ctr Healthcare Operat Improvement & Res CHOIR Drienerlolaan 5 NL-7500 AE Enschede Netherlands Sint Maartensklin Dept Healthcare Logist Hengstdal 3 NL-6574 NA Nijmegen Netherlands Univ Twente Dept Appl Math Stochast Operat Res Drienerlolaan 5 NL-7500 AE Enschede Netherlands Univ Twente Dept Ind Engn & Business Informat Syst IEBIS Drienerlolaan 5 NL-7500 AE Enschede Netherlands Univ Ottawa Telfer Sch Management 55 Laurier Ave East Ottawa ON K1N 6N5 Canada

We study an online capacity planning problem in which arriving patients require a series of appointments at several departments, within a certain access time target. This research is motivated by a study of rehabilitation planning practices at the Sint Maartenskliniek hospital (the Netherlands). In practice, the prescribed treatments and activities are typically booked starting in the first available week, leaving no space for urgent patients who require a series of appointments at a short notice. This leads to the rescheduling of appointments or long access times for urgent patients, which has a negative effect on the quality of care and on patient satisfaction. We propose an approach for allocating capacity to patients at the moment of their arrival, in such a way that the total number of requests booked within their corresponding access time targets is maximized. The model considers online decision making regarding multi-priority, multi-appointment, and multi-resource capacity allocation. We formulate this problem as a Markov decision process (MDP) that takes into account the current patient schedule, and future arrivals. We develop an approximate dynamic programming (ADP) algorithm to obtain approximate optimal capacity allocation policies. We provide insights into the characteristics of the optimal policies and evaluate the performance of the resulting policies using simulation.

关键词： approximate dynamic programming healthcare logistics online capacity planning operations research Markov decision process rehabilitation treatment planning simulation

来源：评论

学校读者我要写书评

暂无评论

Control of a Buck DC/DC Converter Using approximate dynamic programming and Artificial Neural Networks

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS 2021年第4期68卷 1760-1768页

作者： Dong, Weizhen Li, Shuhui Fu, Xingang Li, Zhongwen Fairbank, Michael Gao, Yixiang Univ Alabama Dept Elect & Comp Engn Tuscaloosa AL 35487 USA Texas A&M Univ Dept Elect Engn & Comp Sci Kingsville TX 78363 USA Zhengzhou Univ Sch Elect Engn Zhengzhou 450001 Peoples R China Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England

This paper proposes a novel artificial neural network (ANN) based control method for a dc/dc buck converter. The ANN is trained to implement optimal control based on approximate dynamic programming (ADP). Special characteristics of the proposed ANN control include: 1) The inputs to the ANN contain error signals and integrals of the error signals, enabling the ANN to have PI control ability;2) The ANN receives voltage feedback signals from the dc/dc converter, making the combined system equivalent to a recurrent neural network;3) The ANN is trained to minimize a cost function over a long time horizon, making the ANN have a stronger predictive control ability than a conventional predictive controller;4) The ANN is trained offline, preventing the instability of the network caused by weight adjustments of an on-line training algorithm. The ANN performance is evaluated through simulation and hardware experiments and compared with conventional control methods, which shows that the ANN controller has a strong ability to track rapidly changing reference commands, maintain stable output voltage for a variable load, and manage maximum duty-ratio and current constraints properly.

关键词： Voltage control Buck converters Control systems Switching frequency Transfer functions Inductors Switches dc dc buck converter artificial neural network approximate dynamic programming optimal control

来源：评论

学校读者我要写书评

暂无评论

Stochastic economic dispatch of power system with multiple wind farms and pumped-storage hydro stations using approximate dynamic programming

引用

IET RENEWABLE POWER GENERATION 2020年第13期14卷 2507-2516页

作者： Lin, Shunjiang Fan, Guansheng Jian, Ganyang Liu, Mingbo South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China China Southern Power Grid Elect Power Res Inst Guangzhou 510663 Peoples R China

The stochastic economic dispatch problem of power system with multiple wind farms and pumped-storage hydro stations is formulated as a specific stochastic dynamic programming (DP) model, i.e. stochastic storage model, it is impossible to obtain an accurate solution due to the curse of dimensionality. Based on the approximate DP (ADP) method, the stochastic storage model can be transformed into a series of mixed-integer linear programming (MILP) models by describing the approximate value functions (AVFs) as convex piecewise linear functions in post-decision states. The AVFs are first initialised using the results of the deterministic model under a forecast scenario of wind farm output and then trained by scanning stochastic sampling scenarios consecutively with the successive projective approximation routine algorithm. To obtain a near-optimal day-ahead dispatch scheme, the forecast scenario is substituted into the MILP models expressed by the trained AVFs and is solved forward through each time interval. The network constraints are incorporated by the while-loop detection of critical lines. Test results on an actual provincial power system and the modified IEEE 39-bus system, including the comparison among the ADP, DP, scenario-based and chance-constrained programming methods, demonstrate the feasibility and efficiency of the proposed model and algorithm.

关键词： approximation theory stochastic processes dynamic programming power generation economics wind power plants power generation dispatch pumped-storage power stations integer programming stochastic programming power system security optimisation linear programming multiple wind farms pumped-storage hydro stations approximate dynamic programming stochastic economic dispatch problem specific stochastic dynamic programming model stochastic storage model approximate DP method mixed-integer linear programming models approximate value functions AVFs convex piecewise linear functions deterministic model forecast scenario wind farm output stochastic sampling scenarios successive projective approximation routine algorithm near-optimal day-ahead dispatch scheme MILP models actual provincial power system modified IEEE 39-bus system

来源：评论

学校读者我要写书评

暂无评论

Novel time-space network flow formulation and approximate dynamic programming approach for the crane scheduling in a coil warehouse

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2017年第2期262卷 424-437页

作者： Yuan, Yuan Tang, Lixin Northeastern Univ Inst Ind & Syst Engn Shenyang Peoples R China

This article proposes an efficient event-based time-space network flow model with side constraints for the crane scheduling problem in a coil warehouse where the crane should carry out a set of coil storage, retrieval and shuffling requests, and determine the sequence of handling these requests as well as the positions to which the coils are moved. The model is formulated based on a graph such that each node represents a location in the warehouse at the end of a specific scheduling stage, and each edge indicates a crane's move between two locations in a stage. Variables reduction strategies are presented to accelerate solving the model. In order to solve large-sized instances of the problem, an exact dynamic programming approach based on optimal assignments between coils and positions in a bipartite network with cuts is designed by exploiting the problem structure. Then an approximate dynamic programming (ADP) approach is developed, in which an affine value function approximation is defined as the estimation of crane's traveling time for handling each coil, and updated via iterations by collecting information from the solutions of separate subproblems. Computational results show that the proposed model is tighter and can be solved much more quickly than a traditional model for a reduced crane scheduling problem in the literature and the standard time-space network flow model. Besides, the proposed algorithm can obtain high quality solutions for large-sized instances in a few minutes and is more efficient in solving the problem than a commercial software package. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Scheduling Logistics Coil warehouse Time-space network flow formulation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Sub-optimal switching in anti-lock brake systems using approximate dynamic programming

引用

IET CONTROL THEORY AND APPLICATIONS 2019年第9期13卷 1413-1424页

作者： Sardarmehni, Tohid Heydari, Ali Texas A&M Univ Engn Technol & Ind Distribut College Stn TX 77843 USA Southern Methodist Univ Mech Engn Dallas TX 75205 USA

Optimal scheduling in an anti-lock brake system of ground vehicles is performed through approximate dynamic programming for reducing the stopping distance in severe braking. The proposed optimal scheduler explicitly incorporates the hybrid nature of the anti-lock brake system and provides a feedback solution with a negligible computational burden in control calculation. To this goal, an iterative scheme, called the value iteration algorithm, is used to derive the infinite horizon solution to the underlying Hamilton-Jacobi-Bellman equation. Performance of the proposed method in control of the brake system is illustrated using both linear-in-parameter neural networks and multi-layer perceptrons. Simulation results demonstrate potentials of the method.

关键词： optimal control dynamic programming brakes scheduling feedback braking iterative methods multilayer perceptrons infinite horizon road traffic control vehicle dynamics approximate dynamic programming optimal scheduling anti-lock brake system sub-optimal switching ground vehicles feedback solution value iteration algorithm infinite horizon solution Hamilton-Jacobi-Bellman equation linear-in-parameter neural networks multilayer perceptrons

来源：评论

学校读者我要写书评

暂无评论

Horizontal combinations of online and offline approximate dynamic programming for stochastic dynamic vehicle routing

引用

CENTRAL EUROPEAN JOURNAL OF OPERATIONS RESEARCH 2020年第1期28卷 279-308页

作者： Ulmer, Marlin W. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany

Stochastic and dynamic vehicle routing problems gain increasing attention in the research community. In these problems, routing plans are dynamically updated based on realizations of stochastic information. Due to the complexity of the corresponding Markov decision processes (MDPs), the calculation of optimal policies for these problems is usually not possible and researchers draw on heuristical methods of approximate dynamic programming (ADP). These methods use simulation to approximate the value of a state and decision in the MDP. The simulations are either conducted offline or online. Offline methods such as value function approximations (VFAs) generally neglect the full detail of the state space due to aggregation. Online methods such as rollout algorithms (RAs) are often not able to capture decision and transition space sufficiently due to runtime limitations. In this paper, we alleviate this tradeoff by combining two methods of ADP, an online RA and an offline VFA in two ways. In addition to the integration of the VFA as a base policy into the online RA to strengthen the RA's simulations, we also limit the RA's simulation horizon, estimating the remaining reward-to-go again via the VFA. For two stochastic dynamic routing problems from the literature, we show how this combination outperforms state-of-the-art solutions while simultaneously reducing the required time for online calculations.

关键词： approximate dynamic programming Rollout algorithm Value function approximation dynamic vehicle routing Stochastic requests Multi-period

来源：评论

学校读者我要写书评

暂无评论

Risk-Averse approximate dynamic programming with Quantile-Based Risk Measures

引用

MATHEMATICS OF OPERATIONS RESEARCH 2018年第2期43卷 554-579页

作者： Jiang, Daniel R. Powell, Warren B. Univ Pittsburgh Dept Ind Engn Pittsburgh PA 15261 USA Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08540 USA

In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs;we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measures that includes the popular value at risk (VaR) and the conditional value at risk (CVaR). Although there is considerable theoretical development of risk-averse MDPs in the literature, the computational challenges have not been explored as thoroughly. We propose data-driven and simulation-based approximate dynamic programming (ADP) algorithms to solve the risk-averse sequential decision problem. We address the issue of inefficient sampling for risk applications in simulated settings and present a procedure, based on importance sampling, to direct samples toward the "risky region" as the ADP algorithm progresses. Finally, we show numerical results of our algorithms in the context of an application involving risk-averse bidding for energy storage.

关键词： approximate dynamic programming dynamic risk measures energy trading reinforcement learning Q-learning

来源：评论

学校读者我要写书评

暂无评论

Sourcing strategies in supply risk management: An approximate dynamic programming approach

引用

COMPUTERS & OPERATIONS RESEARCH 2013年第5期40卷 1371-1382页

作者： Fang, Jiarui Zhao, Lei Fransoo, Jan C. Van Woensel, Tom Tsinghua Univ Dept Ind Engn Beijing 100084 Peoples R China Eindhoven Univ Technol Sch Ind Engn NL-5600 MB Eindhoven Netherlands

In recent years, supply chains have become increasingly globalized. As a consequence, the world's supply of all types of parts has become more susceptible to disruptions. Some of these disruptions are extreme and may have global implications. Our research is based on the supply risk management problem faced by a manufacturer. We model the problem as a dynamic program, design and implement approximate dynamic programming (ADP) algorithms to solve it, to overcome the well-known curses of dimensionality. Using numerical experiments, we compare the performance of different ADP algorithms. We then design a series of numerical experiments to study the performance of different sourcing strategies (single, dual, multiple, and contingent sourcing) under various settings, and to discover insights for supply risk management practice. The results show that, under a wide variety of settings, the addition of a third or more suppliers brings much less marginal benefits. Thus, managers can limit their options to a backup supplier (contingent sourcing) or an additional regular supplier (dual sourcing). Our results also show that, unless the backup supplier can supply with zero lead time, using dual sourcing appears to be preferable. Lastly, we demonstrate the capability of the proposed method in analyzing more complicated realistic supply chains. (C) 2012 Elsevier Ltd. All rights reserved.

关键词： Supply disruptions Recurrent risks Sourcing strategies approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：