检索结果-内蒙古大学图书馆

Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands

引用

COMPUTERS & OPERATIONS RESEARCH 2023年第1期159卷

作者： Dastpak, Mohsen Errico, Fausto Jabali, Ola Ecole Technol Super Dept Genie Construct Montreal PQ H3C 1K3 Canada GERAD Montreal PQ H3T 1J4 Canada CIRRELT Montreal PQ H3C 3J7 Canada Politecn Milan Dipartimento Elettron Informaz & Bioingn Piazza Leonardo da Vinci 32 I-20133 Milan Italy

We study a stochastic variant of the vehicle routing problem (VRP) arising in the context of domestic donor collection services. The problem we consider combines the following attributes. Customers requesting services are variable, in the sense that they are stochastic, but are not restricted to a predefined set. Furthermore, demand volumes are also stochastic and are observed upon visiting customers. The objective is to maximize the expected served demands while meeting vehicle capacity and time restrictions. We call this problem the VRP with a highly Variable Customer basis and Stochastic Demands (VRP-VCSD). We first propose a classical Markov Decision Process (MDP) formulation for the VRP-VCSD. The resulting model is, however, unusable due to the explosion in the dimension of the state and action *** solve the VRP-VCSD, we propose a number of methodological contributions aimed at reducing the state and action spaces. We first reformulate the MDP as an MDP with a consecutive action selection procedure. In this formulation, we enforce the treatment of a single vehicle (as opposed to multiple vehicles) at each decision epoch. We then introduce an observation function that selects a subset of the available information, which is deemed relevant for the considered vehicle in each epoch. We develop a Q-learning algorithm called QN-CO. In particular, we use a continuous state representation and incorporate a two-layer artificial neural network to approximate the Q values. Furthermore, we propose an aggregation strategy yielding a fixed-size output. Finally, we enhance our algorithm with Replay Memory and a Double Q *** conduct a thorough computational analysis. Results show that QN-CO considerably outperforms five benchmark policies. Moreover, we show that QN-CO can compete with specialized methods developed for the particular case of the VRP-VCSD where customer locations and expected demands are known in advance.

关键词： Stochastic vehicle routing problem approximate dynamic programming Q-learning

来源：评论

学校读者我要写书评

暂无评论

An approximate dynamic programming approach for comparing firing policies in a networked air defense environment

引用

COMPUTERS & OPERATIONS RESEARCH 2020年第0期117卷 104890-000页

作者： Summers, Daniel S. Robbins, Matthew J. Lunday, Brian J. Air Force Inst Technol Dept Operat Sci Wright Patterson AFB OH 45433 USA

An objective for effective air defense is to identify the firing policy for interceptor allocation to incoming missiles that minimizes the expected total damage to defended assets over a sequence of engagements. We formulate this dynamic weapon target assignment problem as a Markov decision process and utilize a simulation-based, approximate dynamic programming (ADP) approach to solve problem instances based on a representative scenario. Least squares policy evaluation and least squares temporal differences algorithms are developed to determine approximate solutions. A designed experiment investigates problem features such as conflict duration, attacker and defender weapon sophistication, and defended asset values. An empirical comparison of the ADP policies and two baseline policies (i.e., firing either one or two interceptors at each incoming theater ballistic missile (TBM)) yields several insights: the ADP policies outperform both baseline polices when conflict duration is short and attacker weapons are sophisticated;firing one interceptor at each TBM (regardless of inventory status) outperforms the tested ADP policies when conflict duration is long and attacker weapons are less sophisticated;and firing two interceptors at each TBM (regardless of inventory status), which is the United States Army's currently implemented policy, is never the superlative policy for the test instances investigated. Published by Elsevier Ltd.

关键词： Military Air and missile defense dynamic weapon target assignment problem Markov decision processes approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Adaptive Railway Traffic Control using approximate dynamic programming

引用

Transportation Research Procedia 2019年 38卷 201-221页

作者： Taha Ghasempour Benjamin Heydecker Centre for Transport Studies University College London London WC1E 6BT United Kingdom

This study presents an adaptive railway traffic controller for real-time operations based on approximate dynamic programming (ADP). By assessing requirements and opportunities, the controller aims to limit consecutive delays resulting from trains that entered a control area behind schedule by sequencing them at critical locations in a timely manner, thus representing the practical requirements of railway operations. This approach depends on an approximation to the value function of dynamic programming after optimisation from a specified state, which is estimated dynamically from operational experience using reinforcement learning techniques. By using this approximation, the ADP avoids extensive explicit evaluation of performance and so reduces the computational burden substantially. In this investigation, we explore formulations of the approximation function and variants of the learning techniques used to estimate it. Evaluation of the ADP methods in a stochastic simulation environment shows considerable improvements in consecutive delays by comparison with the current industry practice of First-Come-First-Served sequencing. We also found that estimates of parameters of the approximate value function are similar across a range of test scenarios with different mean train entry delays.

关键词： approximate dynamic programming Railway Traffic Management Adaptive Control Reinforcement Learning

来源：评论

学校读者我要写书评

暂无评论

Stochastic economic dispatch of regional integrated energy system considering the pipeline dynamics using improved approximate dynamic programming

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2022年 141卷

作者： Liang, Weikun Lin, Shunjiang Liu, Mingbo Wang, Qiong Xie, Yuquan Sheng, Xuan South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China South China Univ Technol Guangdong Key Lab Clean Energy Technol Guangzhou 511458 Peoples R China

In the regional integrated energy system (RIES), the pipeline dynamics in cooling, heating, and gas networks make their source outputs and load demands difficult to balance in real-time. Hence the economic dispatch problem of RIES requires considering the pipeline dynamics to make the results in line with the actual situation. In this paper, the method of characteristics is proposed to solve the approximate analytical solution (AAS) of partial differential equations describing the gas pipeline dynamics, and the corrected parameters in the AAS are identified by the data-driven method. The high computational accuracy of the obtained AAS is verified by an example. Considering the stochastic fluctuation of renewable energy output and the energy storage ability of cooling, heating, and gas pipelines in a RIES, a stochastic multiple energy storage model for the stochastic economic dispatch problem of RIES considering the pipeline dynamics is established. An improved approximate dynamic programming algorithm considering the coupling of power balance among several adjacent time intervals due to the pipeline dynamics is proposed to efficiently solve the stochastic multiple energy storage model. Test results on an actual RIES, including comparison with the scenario-based method, demonstrate the correctness and efficiency of the proposed model and algorithm.

关键词： RIES Stochastic economic dispatch Pipeline dynamics Method of characteristics Stochastic multiple energy storage model approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Optimal Charging Strategy for EV Charging Stations by Two-Stage approximate dynamic programming

引用

IFAC Proceedings Volumes 2013年第5期46卷 423-430页

作者： Lei Zhang Yaoyu Li Department of Electrical Engineering University of Texas at Dallas Richardson TX 75080 USA Department of Mechanical Engineering University of Texas at Dallas Richardson TX 75080 USA (Tel: 972-883-4698

This study targets the charging scenario for multiple chargeable vehicles at parking lots or charging stations where most vehicles have extended parking time, e.g. at the commercial buildings. All the vehicles are desired to be charged to full prior to the departure time specified, under constrained total charging capacity. A two-stage approximate dynamic programming (TSADP) framework is proposed to determine the optimal charging strategy, utilizing the predicted short-term future information and long-term estimation from historical data. The algorithm is separated into two stages, the optimization stage and approximation stage. The optimization stage is to find the optimal charging strategy for the short-term ahead time horizon. The approximation stage is to approximate the long-term time horizon charging cost, by the way of approximate dynamic programming (ADP). The whole algorithm works as a partial rollout algorithm and partial approximate dynamic programming. The TSADP framework can be integrated into stochastic programming framework, when the short-term information is uncertain with probabilities. The simulation results show that the proposed method can significantly decrease the energy cost.

关键词： Electric Vehicle Charging Control approximate dynamic programming Stochastic programming Prediction

来源：评论

学校读者我要写书评

暂无评论

Insect pest control, approximate dynamic programming, and the management of the evolution of resistance

引用

ECOLOGICAL APPLICATIONS 2019年第2期29卷 1-13页

作者： Hackett, Sean C. Bonsall, Michael B. Dept Zool Math Ecol Res Grp South Parks Rd Oxford OX1 3PS England St Peters Coll New Inn Hall St Oxford OX1 2DL England

Ecological decision problems, such as those encountered in agriculture, often require managing conflicts between short-term costs and long-term benefits. dynamic programming is an ideal method for optimally solving such problems but agricultural problems are often subject to additional complexities that produce state spaces intractable to exact solutions. In contrast, look-ahead policies, a class of approximate dynamic programming (ADP) algorithm, may attempt to solve problems of arbitrary magnitude. However, these algorithms focus on a temporally truncated caricature of the full decision problem over a defined planning horizon and as such are not guaranteed to suggest optimal actions. Thus, look-ahead policies may offer promising means of addressing detail-rich ecological decision problems but may not be capable of fully utilizing the information available to them, especially in scenarios where the best short- and long-term solutions may differ. We constructed and applied look-ahead policies to the management of a hypothetical, stage-structured, continually reproducing, agricultural insect pest. The management objective was to minimize the combined costs of management actions and crop damage over a 16-week growing season. The manager could elect to utilize insecticidal sprays or one of six release ratios of male-selecting transgenic insects where the release ratio determines the number of transgenic insects to be released for each wild-type male insect in the population. Complicating matters was the expression of insecticide resistance at non-trivial frequencies in the pest population. We assessed the extent to which look-ahead policies were able to recognize the potential threat of insecticide resistance and successfully integrate insecticides and transgenic releases to capitalize upon their respective benefits. Look-ahead policies were competent at anticipating and responding to ecological and economic information. Policies with longer planning horizons made fewe

关键词： agricultural insect pests approximate dynamic programming dynamic programming look-ahead policy resistance management stage structure transgenic insect releases

来源：评论

学校读者我要写书评

暂无评论

Accelerated Continuous-Time approximate dynamic programming via Data-Assisted Hybrid Control

引用

IFAC-PapersOnLine 2022年第12期55卷 561-566页

作者： Daniel E. Ochoa Jorge I. Poveda

We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are able to accelerate the convergence properties of the closed-loop system, achieving superior transient performance compared to traditional gradient-descent based techniques. In addition, by leveraging the existence of past recorded data with sufficiently rich information properties, we dispense with the persistence of excitation condition traditionally imposed on the regressors of the critic and the actor. Given that our continuous-time momentum-based dynamics also incorporate periodic discrete-time resets that emulate restarting techniques used in the machine learning literature, we leverage tools from hybrid dynamical systems theory to establish asymptotic stability properties for the closed-loop system. We illustrate our results with a numerical example.

关键词： approximate dynamic programming concurrent learning hybrid systems Lyapunov theory

来源：评论

学校读者我要写书评

暂无评论

Robust Adaptive Control with Active Learning for Fed-Batch Process based on approximate dynamic programming

引用

IFAC-PapersOnLine 2020年第2期53卷 5201-5206页

作者： Ha-Eun Byun Boeun Kim Jay H. Lee Department of Chemical and Biomolecular Engineering Korea Advanced Institute of Science and Technology 291 Daehak-ro Yuseong-gu Daejeon 34141 Republic of Korea Department of Chemical and Biological Engineering University of Wisconsin-Madison Madison WI 53706 USA

Batch process is often subject to a high degree of uncertainty in raw material quality and other initial feedstock conditions. One of the key objectives in operating a batch process is achieving consistent performance and constraint satisfaction in the presence of these uncertainties. This study presents a method for optimal control of a fed-batch process, which can actively and robustly cope with system uncertainty. As in dual control, the method aims to achieve an optimal balance between control actions (exploitation) and probing actions (exploration), leading to improved process performance by actively reducing system uncertainty. An optimal solution of the dual control problem can be found by stochastic dynamic programming but it is computationally intractable in most practical cases. In this study, an approximate dynamic programming (ADP) method for solving the dual control problem is tailored to a batch process which involves non-stationary and nonlinear dynamics. Rewards are formulated to maximize a given end objective while satisfying path constraints. Performance of the ADP-based dual controller is tested on a fed-batch bioreactor with two uncertain parameters.

关键词： Robust adaptive control Dual control Stochastic optimal control approximate dynamic programming Fed-batch process Model uncertainty

来源：评论

学校读者我要写书评

暂无评论

ACCOUNTING RISK IN MULTISTAGE STOCHASTIC PROBLEMS USING approximate dynamic programming

引用

IFAC Proceedings Volumes 2007年第5期40卷 153-158页

作者： Nikolaos E. Pratikakis Matthew J. Realff Jay H. Lee Chemical and Biomolecular Engineering Georgia Institute of Technology311 Ferst Drive Atlanta GA 30332-0100 USA

This work proposes a methodology to generate risk averse policies for Markov Decision Processes(MDPs). This methodology is based on modifying the one stage reward or cost to weigh the trade-off between expected performance and downside risk represented by (CVαR α ). The modified stage-wise utility function is used within dynamic programming to generate a set of policies representing different levels of the trade-off. The approach is demonstrated in a shortest path optimal control problem and a project management problem modeled as constrained MDP. To address a more complex management problem, we utilize the Real Time approximate dynamic programming algorithm.

关键词： Real Time dynamic programming Risk Markov Decision Problems approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

A review of approximate dynamic programming applications within military operations research

引用

OPERATIONS RESEARCH PERSPECTIVES 2021年 8卷

作者： Rempel, M. Cai, J. Def Res & Dev Canada Ctr Operat Res & Anal 101 Colonel By Dr Ottawa ON K1A 0K2 Canada Canadian Joint Operat Command 1600 Star Top Rd Ottawa ON K1B 3W6 Canada

Sequences of decisions that occur under uncertainty arise in a variety of settings, including transportation, communication networks, finance, defence, etc. The classic approach to find an optimal decision policy for a sequential decision problem is dynamic programming;however its usefulness is limited due to the curse of dimensionality and the curse of modelling, and thus many real-world applications require an alternative approach. Within operations research, over the last 25 years the use of approximate dynamic programming (ADP), known as reinforcement learning in many disciplines, to solve these types of problems has increased in popularity. These efforts have resulted in the successful deployment of ADP-generated decision policies for driver scheduling in the trucking industry, locomotive planning and management, and managing high value spare parts in manufacturing. In this article we present the first review of applications of ADP within a defence context, specifically focusing on those which provide decision support to military or civilian leadership. This article's main contributions are twofold. First, we review 18 decision support applications, spanning the spectrum of force development, generation, and employment, that use an ADP-based strategy and for each highlight how its ADP algorithm was designed, evaluated, and the results achieved. Second, based on the trends and gaps identified we discuss five topics relevant to applying ADP to decision support problems within defence: the classes of problems studied;best practices to evaluate ADP-generated policies;advantages of designing policies that are incremental versus complete overhauls when compared to currently practiced policies;the robustness of policies as scenarios change, such as a shift from high to low intensity conflict;and sequential decision problems not yet studied within defence that may benefit from ADP.

关键词： Sequential decision problem Markov decision process approximate dynamic programming Reinforcement learning Military

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：