检索结果-内蒙古大学图书馆

Three bounded proofs for nonlinear multi-input multi-output approximate dynamic programming based on the Lyapunov stability theory

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2018年第1期39卷 35-50页

作者： Huang, Zhijian Xiong, Xuemei Chen, Wentao Zhang, Qin Liu, Yihua Chen, Yuli Shanghai Maritime Univ Merchant Marine Coll Shanghai 201306 Peoples R China Shanghai Jiao Thng Univ Inst Power Plant & Automat Shanghai 200030 Peoples R China

The approximate dynamic programming needs 2 prerequisites to be an effective optimal control method. Firstly, it must be assured to be stable and convergent before application. Secondly, the control system should mainly he a nonlinear multi-input multi-output form. Thus, this paper introduces a nonlinear multi-input multi-output approximate dynamic programming and proves that it is stable in Lyapunov sense, therefore it is convergent. Besides, the Lyapunov function design is also analyzed. These proofs are based on the Lyapunov stability theory in the form of the utility function of quadratic, square-weighted sum, and absolute value. Thereafter, 3 typical control examples of nonlinear multi-input multi-output approximate dynamic programming arc offered to show their applications and verify the proofs. The proof overcomes the complex derivation, and the results contain 3 practical and systematic bounded proofs. It is for the first time that the proof focuses on nonlinear multi-input multi-output approximate dynamic programming from the view of utility function. What is more, the results can also serve as an effective analysis and guide for the utility function design and the stability criterion of nonlinear multi-input multi-output approximate dynamic programming as well.

关键词： approximate dynamic programming bounded proof convergent emission glider idling Lyapunov stability theory multi-input multi-output nonlinear utility function

来源：评论

学校读者我要写书评

暂无评论

Risk-Averse approximate dynamic programming with Quantile-Based Risk Measures

引用

MATHEMATICS OF OPERATIONS RESEARCH 2018年第2期43卷 554-579页

作者： Jiang, Daniel R. Powell, Warren B. Univ Pittsburgh Dept Ind Engn Pittsburgh PA 15261 USA Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08540 USA

In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs;we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measures that includes the popular value at risk (VaR) and the conditional value at risk (CVaR). Although there is considerable theoretical development of risk-averse MDPs in the literature, the computational challenges have not been explored as thoroughly. We propose data-driven and simulation-based approximate dynamic programming (ADP) algorithms to solve the risk-averse sequential decision problem. We address the issue of inefficient sampling for risk applications in simulated settings and present a procedure, based on importance sampling, to direct samples toward the "risky region" as the ADP algorithm progresses. Finally, we show numerical results of our algorithms in the context of an application involving risk-averse bidding for energy storage.

关键词： approximate dynamic programming dynamic risk measures energy trading reinforcement learning Q-learning

来源：评论

学校读者我要写书评

暂无评论

Optimized radial basis function neural network for improving approximate dynamic programming in pricing high dimensional options

引用

NEURAL COMPUTING & APPLICATIONS 2018年第6期30卷 1783-1794页

作者： Hajizadeh, Ehsan Mahootchi, Masoud Amirkabir Univ Technol Tehran Polytech Dept Ind Engn & Management Syst 424 Hafez Ave Tehran *** Iran

Pricing American basket option is one of the essential problems in quantitative finance. The complexity of this type of option has motivated many practitioners and researchers to develop simulation-based methods. In this paper, we develop an optimized radial basis function neural network (RBFNN), which is optimally tuned by the particle swarm optimization algorithm to enhance the efficiency and accuracy of approximate dynamic programming (ADP) for pricing the American basket option. Additionally, for the scenario generation, a simulation-based technique using a copula-GARCH method and Extreme Value Theory is performed to tackle the nonlinearity of dependencies between variables. The prices obtained through the proposed approach compared with those ones achieved from pure RBFNN and ADP in different situations. This is also illustrated that the obtained prices of American basket option can outperform the results obtained through the RBFNN and ADP approaches in terms of the predefined fitness measures.

关键词： Radial basis function neural network approximate dynamic programming Particle swarm optimization algorithm American basket options

来源：评论

学校读者我要写书评

暂无评论

Vehicle scheduling under stochastic trip times: An approximate dynamic programming approach

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2018年 96卷 144-159页

作者： He, Fang Yang, Jie Li, Meng Tsinghua Univ Dept Ind Engn Beijing 100084 Peoples R China Tsinghua Univ Tsinghua Daimler Joint Res Ctr Sustainable Transp Beijing 100084 Peoples R China Tsinghua Univ Dept Civil Engn Beijing 100084 Peoples R China

Due to unexpected demand surge and supply disruptions, road traffic conditions could exhibit substantial uncertainty, which often makes bus travelers encounter start delays of service trips and substantially degrades the performance of an urban transit system. Meanwhile, rapid advances of information and communication technologies have presented tremendous opportunities for intelligently scheduling a bus fleet. With the full consideration of delay propagation effects, this paper is devoted to formulating the stochastic dynamic vehicle scheduling problem, which dynamically schedules an urban bus fleet to tackle the trip time stochasticity, reduce the delay and minimize the total costs of a transit system. To address the challenge of "curse of dimensionality", we adopt an approximate dynamic programming approach (ADP) where the value function is approximated through a three-layer feed-forward neural network so that we are capable of stepping forward to make decisions and solving the Bellman's equation through sequentially solving multiple mixed integer linear programs. Numerical examples based on the realistic operations dataset of bus lines in Beijing have demonstrated that the proposed neural network -based ADP approach not only exhibits a good learning behavior but also significantly outperforms both myopic and static polices, especially when trip time stochasticity is high.

关键词： Vehicle scheduling Stochastic trip times Delay propagation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

An approximate dynamic programming method for the optimal control of Alkai-Surfactant-Polymer flooding

引用

JOURNAL OF PROCESS CONTROL 2018年 64卷 15-26页

作者： Ge, Yulei Li, Shurong Chan, Peng China Univ Petr East China Coll Informat & Control Engn Qingdao 266580 Peoples R China Beijing Univ Posts & Telecommun Automat Sch Beijing 100876 Peoples R China Jiangsu Automat Res Inst Lianyungang 222006 Peoples R China

Since the complexity, coupling, distributed parameter, etc. of alkali-surfactant-polymer (ASP) flooding, common optimization methods cannot acquire the optimal solutions well. This paper brings an optimal control method for ASP flooding based on approximate dynamic programming (ADP). At first, take the net present value (NPV) as the performance index. Then the Actor-Critic algorithm based on gradient descent method is adopted to get the optimal injection strategy, in which Actor and Critic are used to approximate the control and value function, respectively. To improve the approximation performance, the linear approximation basis function based on system characteristic is constructed. Furthermore, to train and predict the control and value function in next step, a temporal difference (TD) learning algorithm is introduced to update the weight coefficients. Then, the control in ADP is generated according to the Gauss function and its weight is updated according to the sigmoid function of TD error, so that the optimal control can be searched. At last, the enhanced oil recovery problem of ASP flooding with four injection wells and nine production wells is solved by the proposed method to test the effect of proposed method. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： ASP flooding approximate dynamic programming Actor-Critic algorithm Temporal difference learning algorithm TD error Linear basis function

来源：评论

学校读者我要写书评

暂无评论

FROM INFINITE TO FINITE PROGRAMS: EXPLICIT ERROR BOUNDS WITH APPLICATIONS TO approximate dynamic programming

引用

SIAM JOURNAL ON OPTIMIZATION 2018年第3期28卷 1968-1998页

作者： Esfahani, Peyman Mohajerin Sutter, Tobias Kuhn, Daniel Lygeros, John Delft Univ Technol Delft Ctr Syst & Control Delft Netherlands Swiss Fed Inst Technol Automat Control Lab Zurich Switzerland Ecole Polytech Fed Lausanne Risk Analyt & Optimizat Chair Lausanne Switzerland

We consider linear programming (LP) problems in infinite dimensional spaces that are in general computationally intractable. Under suitable assumptions, we develop an approximation bridge from the infinite dimensional LP to tractable finite convex programs in which the performance of the approximation is quantified explicitly. To this end, we adopt the recent developments in two areas of randomized optimization and first-order methods, leading to a priori as well as a posteriori performance guarantees. We illustrate the generality and implications of our theoretical results in the special case of the long-run average cost and discounted cost optimal control problems in the context of Markov decision processes on Borel spaces. The applicability of the theoretical results is demonstrated through a fisheries management problem.

关键词： infinite dimensional linear programming Markov decision processes approximate dynamic programming randomized and convex optimization

来源：评论

学校读者我要写书评

暂无评论

Stochastic dynamic resource allocation for HIV prevention and treatment: An approximate dynamic programming approach

引用

COMPUTERS & INDUSTRIAL ENGINEERING 2018年 118卷 423-439页

作者： Cosgun, Ozlem Buyuktahtakin, I. Esra Wichita State Univ Dept Ind & Mfg Engn Wichita KS 67260 USA New Jersey Inst Technol Dept Mech & Ind Engn Newark NJ 07102 USA

Human immunodeficiency virus (HIV) is a key global health concern, with 33 million people living with HIV worldwide and 2.7 million new infections occurring annually. To prevent the spread of this widely prevalent epidemic disease, prevention and treatment intervention strategies urgently need to be implemented. The goal of this study is to propose stochastic dynamic programming (SDP) and approximate dynamic programming (ADP) algorithms that will optimally allocate the limited intervention budget among the HIV disease compartments and determine the best set of interventions that should be applied to each disease compartment, while minimizing the number of HIV-infected and people diagnosed with acquired immune deficiency syndrome (AIDS) as well as related deaths over a multi-year planning horizon. A compartmental model is constructed and formulated as a nonstationary Markov decision process (MDP) in order to capture the progression of the disease among the highest risk group-African American/black men who have sex with men (BMSM). In order to alleviate the computational difficulties arising from the exponentially growing state space in the SDP, we propose ADP algorithms that determine the approximately optimal budget allocation policies over six years. Our results suggest a greater allocation of the limited budget to prevention measures rather than treatment interventions, such as antiretroviral therapy (ART). As opposed to traditional policies that allocate the budget only once at the beginning of the time horizon, the ADP model suggests using a dynamic proportional budget strategy, allocating the budget dynamically over a multi-period planning period as the uncertainty in disease transmission is revealed. Results show that our ADP approach provides significant increases in health benefits and cost savings in HIV prevention and intervention.

关键词： HIV epidemic disease Resource allocation Interventions approximate dynamic programming Stochastic dynamic programming Disease transmission uncertainty Markov decision process (MDP)

来源：评论

学校读者我要写书评

暂无评论

Adaptive Railway Traffic Control using approximate dynamic programming

引用

Transportation Research Procedia 2019年 38卷 201-221页

作者： Taha Ghasempour Benjamin Heydecker Centre for Transport Studies University College London London WC1E 6BT United Kingdom

This study presents an adaptive railway traffic controller for real-time operations based on approximate dynamic programming (ADP). By assessing requirements and opportunities, the controller aims to limit consecutive delays resulting from trains that entered a control area behind schedule by sequencing them at critical locations in a timely manner, thus representing the practical requirements of railway operations. This approach depends on an approximation to the value function of dynamic programming after optimisation from a specified state, which is estimated dynamically from operational experience using reinforcement learning techniques. By using this approximation, the ADP avoids extensive explicit evaluation of performance and so reduces the computational burden substantially. In this investigation, we explore formulations of the approximation function and variants of the learning techniques used to estimate it. Evaluation of the ADP methods in a stochastic simulation environment shows considerable improvements in consecutive delays by comparison with the current industry practice of First-Come-First-Served sequencing. We also found that estimates of parameters of the approximate value function are similar across a range of test scenarios with different mean train entry delays.

关键词： approximate dynamic programming Railway Traffic Management Adaptive Control Reinforcement Learning

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for missile defense interceptor fire control

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2017年第3期259卷 873-886页

作者： Davis, Michael T. Robbins, Matthew J. Lunday, Brian J. Air Force Inst Technol Dept Operat Sci 2950 Hobson Way Wright Patterson AFB OH 45433 USA

Given the ubiquitous nature of both offensive and defensive missile systems, the catastrophe-causing potential they represent, and the limited resources available to countries for missile defense, optimizing the defensive response to a missile attack is a necessary national security endeavor. For a single salvo of offensive missiles launched at a set of targets, a missile defense system protecting those targets must determine how many interceptors to fire at each incoming missile. Since such missile engagements often involve the firing of more than one attack salvo, we develop a Markov decision process (MDP) model to examine the optimal fire control policy for the defender. Due to the computational intractability of using exact methods for all but the smallest problem instances, we utilize an approximate dynamic programming (ADP) approach to explore the efficacy of applying approximate methods to the problem. We obtain policy insights by analyzing subsets of the state space that reflect a range of possible defender interceptor inventories. Testing of four instances derived from a representative planning scenario demonstrates that the ADP policy provides high -quality decisions for a majority of the state space, achieving a 7.74% mean optimality gap over all states for the most realistic instance, modeling a longer -term engagement by an attacker who assesses the success of each salvo before launching a subsequent one. Moreover, the ADP algorithm requires only a few minutes of computational effort versus hours for the exact dynamic programming algorithm, providing a method to address more complex and realistically-sized instances. Published by Elsevier B.V.

关键词： approximate dynamic programming Least squares temporal differences Markov decision processes Military applications Weapon target assignment problem

来源：评论

学校读者我要写书评

暂无评论

Insect pest control, approximate dynamic programming, and the management of the evolution of resistance

引用

ECOLOGICAL APPLICATIONS 2019年第2期29卷 1-13页

作者： Hackett, Sean C. Bonsall, Michael B. Dept Zool Math Ecol Res Grp South Parks Rd Oxford OX1 3PS England St Peters Coll New Inn Hall St Oxford OX1 2DL England

Ecological decision problems, such as those encountered in agriculture, often require managing conflicts between short-term costs and long-term benefits. dynamic programming is an ideal method for optimally solving such problems but agricultural problems are often subject to additional complexities that produce state spaces intractable to exact solutions. In contrast, look-ahead policies, a class of approximate dynamic programming (ADP) algorithm, may attempt to solve problems of arbitrary magnitude. However, these algorithms focus on a temporally truncated caricature of the full decision problem over a defined planning horizon and as such are not guaranteed to suggest optimal actions. Thus, look-ahead policies may offer promising means of addressing detail-rich ecological decision problems but may not be capable of fully utilizing the information available to them, especially in scenarios where the best short- and long-term solutions may differ. We constructed and applied look-ahead policies to the management of a hypothetical, stage-structured, continually reproducing, agricultural insect pest. The management objective was to minimize the combined costs of management actions and crop damage over a 16-week growing season. The manager could elect to utilize insecticidal sprays or one of six release ratios of male-selecting transgenic insects where the release ratio determines the number of transgenic insects to be released for each wild-type male insect in the population. Complicating matters was the expression of insecticide resistance at non-trivial frequencies in the pest population. We assessed the extent to which look-ahead policies were able to recognize the potential threat of insecticide resistance and successfully integrate insecticides and transgenic releases to capitalize upon their respective benefits. Look-ahead policies were competent at anticipating and responding to ecological and economic information. Policies with longer planning horizons made fewe

关键词： agricultural insect pests approximate dynamic programming dynamic programming look-ahead policy resistance management stage structure transgenic insect releases

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：