检索结果-内蒙古大学图书馆

approximate dynamic programming Methodology for Data-based Optimal Controllers

REVISTA IBEROAMERICANA DE AUTOMATICA E INFORMATICA INDUSTRIAL 2019年第3期16卷 273-283页

作者： Diaz, Henry Armesto, Leopoldo Sala, Antonio Univ Politecn Valencia Inst U Automat & Informat Ind Cno Vera S-N E-46022 Valencia Spain Univ Politecn Valencia Inst Diseno & Fabricac Cno Vera S-N E-46022 Valencia Spain

In this article, we present a methodology for learning data-based approximately optimal controllers, within the context of learning and approximate dynamic programming. There are previous solutions in dynamic programming that use linear programming in discrete state space, but cannot be applied directly to continuous space. The objective of the methodology is to calculate data-based optimal controllers for continuous state space, these controllers are obtained by a lower estimation of the accumulated cost through functional approximators with linear parameterization. This is solved non-iteratively with linear programming, but it requires to provide appropriate conditions for regressor regularization and to introduce a cost of leaving the region with valid data, in order to obtain satisfactory results (avoiding unrestricted or poorly conditioned solutions).

关键词： Intelligent Control approximate dynamic programming Neural Learning Optimal Control

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming with a fuzzy parameterization

引用

AUTOMATICA 2010年第5期46卷 804-814页

作者： Busoniu, Lucian Ernst, Damien De Schutter, Bart Babuska, Robert Delft Univ Technol Delft Ctr Syst &Control NL-2628 CD Delft Netherlands Univ Liege Inst Montefiore FNRS B-4000 Liege Belgium

dynamic programming (DP) is a powerful paradigm for general, nonlinear optimal control. Computing exact DP solutions is in general only possible when the process states and the control actions take values in a small discrete set. In practice, it is necessary to approximate the solutions. Therefore, we propose an algorithm for approximate DP that relies on a fuzzy partition of the state space, and on a discretization of the action space. This fuzzy Q-iteration algorithm works for deterministic processes, under the discounted return criterion. We prove that fuzzy Q-iteration asymptotically converges to a solution that lies within a bound of the optimal solution. A bound on the suboptimality of the solution obtained in a finite number of iterations is also derived. Under continuity assumptions on the dynamics and on the reward function, we show that fuzzy Q-iteration is consistent, i.e., that it asymptotically obtains the optimal solution as the approximation accuracy increases. These properties hold both when the parameters of the approximator are updated in a synchronous fashion, and when they are updated asynchronously. The asynchronous algorithm is proven to converge at least as fast as the synchronous one. The performance of fuzzy Q-iteration is illustrated in a two-link manipulator control problem. (C) 2010 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Fuzzy approximation Value iteration Convergence analysis

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for Military Medical Evacuation Dispatching Policies

引用

INFORMS JOURNAL ON COMPUTING 2021年第1期33卷 2-26页

作者： Jenkins, Phillip R. Robbins, Matthew J. Lunday, Brian J. Air Force Inst Technol Dept Operat Sci Wright Patterson AFB OH 45433 USA

Military medical planners must consider how aerial medical evacuation (MEDEVAC) assets will be dispatched when preparing for and supporting high-intensity combat operations. The dispatching authority seeks to dispatch MEDEVAC assets to prioritized requests for service, such that battlefield casualties are effectively and efficiently transported to nearby medical-treatment facilities. We formulate and solve a discounted, infinite-horizon Markov decision process (MDP) model of the MEDEVAC dispatching problem. Because the high dimensionality and uncountable state space of our MDP model renders classical dynamic programming solution methods intractable, we instead apply approximate dynamic programming (ADP) solution methods to produce high-quality dispatching policies relative to the currently practiced closest-available dispatching policy. We develop, test, and compare two distinct ADP solution techniques, both of which utilize an approximate policy iteration (API) algorithmic framework. The first algorithm uses least-squares temporal differences (LSTD) learning for policy evaluation, whereas the second algorithm uses neural network (NN) learning. We construct a notional, yet representative planning scenario based on high-intensity combat operations in southern Azerbaijan to demonstrate the applicability of our MDP model and to compare the efficacies of our proposed ADP solution techniques. We generate 30 problem instances via a designed experiment to examine how selected problem features and algorithmic features affect the quality of solutions attained by our ADP policies. Results show that the respective policies determined by the NN-API and LSTD-API algorithms significantly outperform the closest-available benchmark policies in 27 (90%) and 24 (80%) of the problem instances examined. Moreover, the NN-API policies significantly outperform the LSTD-API policies in each of the problem instances examined. Compared with the closest-available policy for the baseline probl

关键词： military MEDEVAC EMS system Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming algorithms for multidimensional flexible production-inventory problems

引用

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH 2017年第7期55卷 2034-2050页

作者： Cimen, Mustafa Kirkbride, Chris Hacettepe Univ Dept Business Adm Ankara Turkey Univ Lancaster Sch Management Dept Management Sci Lancaster England

An important issue in the manufacturing and supply chain literature concerns the optimisation of inventory decisions. Single-product inventory problems are widely studied and have been optimally solved under a variety of assumptions and settings. However, as systems become more complex, inventory decisions become more complicated for which the methods/approaches for optimising single inventory systems are incapable of deriving optimal policies. Manufacturing process flexibility provides an example of such a complex application area. Decisions involving the interrelated product inventories and production facilities form a highly multidimensional, non-decomposable system for which optimal policies cannot be readily obtained. We propose the methodology of approximate dynamic programming ( ADP) to overcome the computational challenge imposed by this multidimensionality. Incorporating a sample backup simulation approach, ADP develops policies by utilising only a fraction of the computations required by classical dynamic programming. However, there are few studies in the literature that optimise production decisions in a stochastic, multi-factory, multi-product inventory system of this complexity. This paper aims to explore the feasibility and relevancy of ADP algorithms for this application. We present the results from numerical experiments that establish the strong performance of policies developed via temporal difference ADP algorithms in comparison to optimal policies and to policies derived from a deterministic approximation of the problem.

关键词： approximate dynamic programming machine learning dynamic programming flexible manufacturing process flexibility inventory control

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming with state aggregation applied to UAV perimeter patrol

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2011年第12期21卷 1396-1409页

作者： Krishnamoorthy, K. Pachter, M. Darbha, S. Chandler, P. USAF Control Design & Anal Branch Res Lab RBCA Wright Patterson AFB OH 45433 USA USAF Inst Technol Dept Elect Engn Wright Patterson AFB OH 45433 USA Texas A&M Univ Dept Mech Engn College Stn TX 77843 USA

One encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for large-scale controlled Markov chains. In this paper, we provide a reward-based aggregation method to construct suboptimal policies for a perimeter surveillance control problem which gives rise to a large scale Markov chain. The novelty of this approach lies in circumventing the need for value iteration over the entire state space. Instead, the state space is partitioned and the value function is approximated by a constant over each partition. We associate a meta-state with each partition, where the transition probabilities between these meta-states are known. The state aggregation approach results in a significant reduction in the computational burden and lends itself to value iteration over the aggregated state-space. We provide bounds to assess the quality of the approximation and give numerical results that support the proposed methodology. Published 2011. This article is a US Government work and is in the public domain in the USA.

关键词： controlled Markov chain perimeter patrol approximate dynamic programming state aggregation

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming With Enhanced Off-Policy Learning for Coordinating Distributed Energy Resources

引用

IEEE TRANSACTIONS ON SUSTAINABLE ENERGY 2024年第3期15卷 1614-1626页

作者： Das, Avijit Wu, Di Bhatti, Bilal Ahmad Kamaludeen, Mohamed Pacific Northwest Natl Lab Elect Infrastruct & Bldg Div Richland WA 99352 USA US DOE Washington DC 20585 USA Columbia Univ New York NY 10027 USA

This paper proposes an innovative approximate dynamic programming (ADP) method for distributed energy resource coordination with the loss of life of battery energy storage system (BESS) explicitly modeled. The dispatch policy is designed to account for both calendrical and cyclical aging effects on BESS, explicitly modeling the impacts of ambient temperature on BESS lifespan. The proposed ADP employs an adaptive critic method and enhanced off-policy deterministic policy gradient (DPG) strategy, addressing the limitations of the on-policy gradient-based ADP approaches, including inadequate exploration, low data usage, and computational complexity. In particular, a customized policy is proposed to guide the algorithm to explore some promising decisions and thereby improve exploration capability and learning efficiency compared to conventional DPG-based learning approaches, which may struggle to find a global optimum due to random noisy action-based exploration or require expert demonstration with extra effort. The proposed method is illustrated using the IEEE 123-node system and compared with the existing ADP methods to prove solution accuracy and demonstrate the effects of incorporating degradation models into control design. Case studies showed that the proposed ADP effectively coordinates DERs with a 10 times smaller optimization gap compared to existing methods, and the incorporation of the BESS life loss model into the proposed control ensures the expected lifespan and results in significant cost savings.

关键词： approximate dynamic programming customized exploration deterministic policy gradient distributed energy resources energy storage

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for container stacking

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2023年第1期310卷 328-342页

作者： Boschma, Rene Mes, Martijn R. K. de Vries, Leon R. Univ Twente Fac Elect Engn Math & Comp Sci Drienerlolaan 5 NL-7522 NB Enschede Netherlands Univ Twente Fac Behav Management & Social Sci Drienerlolaan 5 NL-7522 NB Enschede Netherlands

At some point during transport, intermodal containers will be stored at a terminal, where they are typi-cally stacked on top of each other. Stacking yields a higher utilization of the area but may lead to unpro-ductive reshuffle moves when containers below another need to be retrieved. Preventing reshuffles has a financial benefit, as it not only avoids the costs of executing the reshuffle but also decreases the time needed to retrieve a container. Typically, researchers consider only the retrieval of containers and assume the retrieval order is fully known. In addition, existing studies do not consider the stacking restrictions imposed by a reach stacker, which is commonly used in smaller inland terminals. This research aims to design decision support for determining real-life applicable container stack allocations so that the ex-pected number of reshuffles is minimized. We propose a model that includes both arrivals and departures of containers as well as a certain level of uncertainty in the order thereof. The problem is modeled as a Markov Decision Process and solved using approximate dynamic programming (ADP). Through numerical experiments on real-life problem instances, we conclude that the ADP approach drastically outperforms a benchmark heuristic from the literature. All data used as well as the source code has been made publicly available. (c) 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license ( http://***/licenses/by/4.0/ )

关键词： OR In maritime industry Container stacking Container relocation problem approximate dynamic programming Intermodal transport

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming-based approaches for input-output data-driven control of nonlinear processes

引用

AUTOMATICA 2005年第7期41卷 1281-1288页

作者： Lee, JM Lee, JH Georgia Inst Technol Sch Chem & Biomol Engn Atlanta GA 30332 USA

We propose two approximate dynamic programming (ADP)-based strategies for control of nonlinear processes using input-output data. In the first strategy, which we term 'J-Iearning,' one builds an empirical nonlinear model using closed-loop test data and performs dynamic programming with it to derive an improved control policy. In the second strategy, called 'Q-learning,' one tries to learn an improved control policy in a model-less manner. Compared to the conventional model predictive control approach, the new approach offers some practical advantages in using nonlinear empirical models for process control. Besides the potential reduction in the on-line computational burden, it offers a convenient way to control the degree of model extrapolation in the calculation of optimal control moves. One major difficulty associated with using an empirical model within the multi-step predictive control setting is that the model can be excessively extrapolated into regions of the state space where identification data were scarce or nonexistent, leading to performances far worse than predicted by the model. Within the proposed ADP-based strategies, this problem is handled by imposing a penalty term designed on the basis of local data distribution. A CSTR example is provided to illustrate the proposed approaches. (c) 2005 Elsevier Ltd. All rights reserved.

关键词： nonlinear model identification nonlinear model predictive control approximate dynamic programming NARX model reinforcement learning Q-learning

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for the military inventory routing problem

引用

ANNALS OF OPERATIONS RESEARCH 2020年第1期288卷 391-416页

作者： McKenna, Rebekah S. Robbins, Matthew J. Lunday, Brian J. McCormack, Ian M. Air Force Inst Technol Dept Operat Sci 2950 Hobson Way Wright Patterson AFB OH 45433 USA

The United States Army can benefit from effectively utilizing cargo unmanned aerial vehicles (CUAVs) to perform resupply operations in combat environments to reduce the use of manned (ground and aerial) resupply that incurs risk to personnel. We formulate a Markov decision process (MDP) model of an inventory routing problem (IRP) with vehicle loss and direct delivery, which we label the military IRP (MILIRP). The objective of the MILIRP is to determine CUAV dispatching and routing policies for the resupply of geographically dispersed units operating in an austere, combat environment. The large size of the problem instance motivating this research renders dynamic programming algorithms inappropriate, so we utilize approximate dynamic programming (ADP) methods to attain improved policies (relative to a benchmark policy) via an approximate policy iteration algorithmic strategy utilizing least squares temporal differencing for policy evaluation. We examine a representative problem instance motivated by resupply operations experienced by the United States Army in Afghanistan both to demonstrate the applicability of our MDP model and to examine the efficacy of our proposed ADP solution methodology. A designed computational experiment enables the examination of selected problem features and algorithmic features vis-a-vis the quality of solutions attained by our ADP policies. Results indicate that a 4-crew, 8-CUAV unit is able to resupply 57% of the demand from an 800-person organization over a 3-month time horizon when using the ADP policy, a notable improvement over the 18% attained using a benchmark policy. Such results inform the development of procedures governing the design, development, and utilization of CUAV assets for the resupply of dispersed ground combat forces.

关键词： Inventory routing problem Markov decision processes approximate dynamic programming Least squares temporal differences Military

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming strategies and their applicability for process control: A review and future directions

引用

INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS 2004年第3期2卷 263-278页

作者： Lee, JM Lee, JH Georgia Inst Technol Sch Chem & Biomol Engn Atlanta GA 30332 USA

This paper reviews dynamic programming (DP), surveys approximate solution methods for it, and considers their applicability to process control problems. Reinforcement Learning (RL) and Neuro-dynamic programming (NDP), which can be viewed as approximate DP techniques, are already established techniques for solving difficult multi-stage decision problems in the fields of operations research, computer science, and robotics. Owing to the significant disparity of problem formulations and objective, however, the algorithms and techniques available from these fields are not directly applicable to process control problems, and reformulations based on accurate understanding of these techniques are needed. We categorize the currently available approximate solution techniques for dynamic programming and identify those most suitable for process control problems. Several open issues are also identified and discussed.

关键词： approximate dynamic programming reinforcement learning neuro-dynamic programming optimal control function approximation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：