检索结果-内蒙古大学图书馆

Discrete-Time Optimal Control via Local Policy Iteration Adaptive dynamic programming

IEEE TRANSACTIONS ON CYBERNETICS 2017年第10期47卷 3367-3379页

作者： Wei, Qinglai Liu, Derong Lin, Qiao Song, Ruizhuo Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China

In this paper, a discrete-time optimal control scheme is developed via a novel local policy iteration adaptive dynamic programming algorithm. In the discrete-time local policy iteration algorithm, the iterative value function and iterative control law can be updated in a subset of the state space, where the computational burden is relaxed compared with the traditional policy iteration algorithm. Convergence properties of the local policy iteration algorithm are presented to show that the iterative value function is monotonically nonincreasing and converges to the optimum under some mild conditions. The admissibility of the iterative control law is proven, which shows that the control system can be stabilized under any of the iterative control laws, even if the iterative control law is updated in a subset of the state space. Finally, two simulation examples are given to illustrate the performance of the developed method.

关键词： Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming local policy iteration neuro-dynamic programming nonlinear systems optimal control

来源：评论

学校读者我要写书评

暂无评论

dynamic pricing and reservation for intelligent urban parking management

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2017年 77卷 226-244页

作者： Lei, Chao Ouyang, Yanfeng Univ Illinois Dept Civil & Environm Engn Urbana IL 61801 USA

Despite rapid advances of information technologies for intelligent parking systems, it remains a challenge to optimally manage limited parking resources in busy urban neighborhoods. In this paper, we use dynamic location-dependent parking pricing and reservation to improve system-wide performance of an intelligent parking system. With this system, the parking agency is able to decide the spatial and temporal distribution of parking prices to achieve a variety of objectives, while drivers with different origins and destinations compete for limited parking spaces via online reservation. We develop a multi period non-cooperative bi-level model to capture the complex interactions among the parking agency and multiple drivers, as well as a non-myopic approximate dynamic programming (ADP) approach to solve the model. It is shown with numerical examples that the ADP-based pricing policy consistently outperforms alternative policies in achieving greater performance of the parking system, and shows reliability in handling the spatial and temporal variations in parking demand. (C) 2017 Elsevier Ltd. All rights reserved.

关键词： Parking management dynamic pricing approximate dynamic programming Equilibrium MPEC

来源：评论

学校读者我要写书评

暂无评论

dynamic Patient Scheduling for Multi-Appointment Health Care Programs

引用

PRODUCTION AND OPERATIONS MANAGEMENT 2018年第1期27卷 58-79页

作者： Diamant, Adam Milner, Joseph Quereshy, Fayez York Univ Schulich Sch Business 111 Ian Macdonald Blvd Toronto ON M3J 1P3 Canada Univ Toronto Rotman Sch Management 105 St George St Toronto ON M5S 3E6 Canada Toronto Western Hosp Gen Surg 399 Bathurst St Toronto ON M5T 2S8 Canada

We investigate the scheduling practices of a multidisciplinary, multistage, outpatient health care program. Patients undergo a series of assessments before being eligible for elective surgery. Such systems often suffer from high rates of attrition and appointment no-shows leading to capacity underutilization and treatment delays. We propose a new scheduling model where the clinic assigns patients to an appointment day but postpones the decision of which assessments patients undergo pending the observation of who arrives. In doing so, the clinic gains flexibility to improve system performance. We formulate the scheduling problem as a Markov decision process and use approximate dynamic programming to solve it. We apply our approach to a dataset collected from a bariatric surgery program at a large tertiary hospital in Toronto, Canada. We examine the quality of our solutions via structural results and compare them with heuristic scheduling practices using a discrete-event simulation. By allowing multiple assessments, delaying their scheduling, and by optimizing over an appointment book, we show significant improvements in patient throughput, clinic profit, use of overtime, and staff utilization.

关键词： service systems appointment scheduling approximate dynamic programming no-shows multiple appointments

来源：评论

学校读者我要写书评

暂无评论

Natural gas storage valuation via least squares Monte Carlo and support vector regression

引用

ENERGY SYSTEMS-OPTIMIZATION MODELING SIMULATION AND ECONOMIC ASPECTS 2017年第4期8卷 815-855页

作者： Malyscheff, Alexander M. Trafalis, Theodore B. Univ Oklahoma Sch Elect & Comp Engn 110 W Boyd Norman OK 73019 USA Univ Oklahoma Sch Ind & Syst Engn 202 W Boyd Norman OK 73019 USA

Least squares Monte Carlo (LSMC) approaches represent a computationally efficient method for the valuation of natural gas storage facilities. LSMC methods are computationally tractable while they simultaneously allow for a decoupling of tlie price path simulation from the optimization of the decision vector. However, selecting the appropriate features using traditional regression techniques can be challenging, particularly when several factors of uncertainty are assumed to drive the price process. In this paper we analyze a natural gas storage contract using a two factor forward model whose parameters can be easily calibrated. For a forward curve derived from monthly averages of the NBP day-ahead contract from 2004 to 2009 we compute storage values based on a collection of spot price paths and price paths of a daily forward contract with a time to maturity of 30 days. We study the impact of additional pricing information in the form of a forward contract on the value of a gas storage facility. A comparison to the corresponding one factor model is also included in our experiments. Value function approximation is carried out by employing a kernel-based regression technique in the form of support vector machine regression (SVR). We report out-of-sample results by simulating the targets for the next stage. We also carry out a search in the space of SVR parameters to identify the appropriate parameters for our experiments. Applying a spot trading strategy we observe a higher storage value for the one factor model when compared to the corresponding two factor model. With respect to the two factor model we report that an approximation of the value function over both a spot and a forward contract increases storage value compared to a value function that is computed over a spot contract only.

关键词： Natural gas storage valuation Least squares Monte Carlo approximate dynamic programming Support vector machine regression Kernel methods

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming with a fuzzy parameterization

引用

AUTOMATICA 2010年第5期46卷 804-814页

作者： Busoniu, Lucian Ernst, Damien De Schutter, Bart Babuska, Robert Delft Univ Technol Delft Ctr Syst &Control NL-2628 CD Delft Netherlands Univ Liege Inst Montefiore FNRS B-4000 Liege Belgium

dynamic programming (DP) is a powerful paradigm for general, nonlinear optimal control. Computing exact DP solutions is in general only possible when the process states and the control actions take values in a small discrete set. In practice, it is necessary to approximate the solutions. Therefore, we propose an algorithm for approximate DP that relies on a fuzzy partition of the state space, and on a discretization of the action space. This fuzzy Q-iteration algorithm works for deterministic processes, under the discounted return criterion. We prove that fuzzy Q-iteration asymptotically converges to a solution that lies within a bound of the optimal solution. A bound on the suboptimality of the solution obtained in a finite number of iterations is also derived. Under continuity assumptions on the dynamics and on the reward function, we show that fuzzy Q-iteration is consistent, i.e., that it asymptotically obtains the optimal solution as the approximation accuracy increases. These properties hold both when the parameters of the approximator are updated in a synchronous fashion, and when they are updated asynchronously. The asynchronous algorithm is proven to converge at least as fast as the synchronous one. The performance of fuzzy Q-iteration is illustrated in a two-link manipulator control problem. (C) 2010 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Fuzzy approximation Value iteration Convergence analysis

来源：评论

学校读者我要写书评

暂无评论

Spacecraft Autonomy modeled via Markov Decision Process and Associative Rule-based Machine Learning 4

Spacecraft Autonomy modeled via Markov Decision Process and ...

引用

4th IEEE International Workshop on Metrology for AeroSpace

作者： D'Angelo, Gianni Tipaldi, Massimo Glielmo, Luigi Rampone, Salvatore Univ Sannio Dept Sci & Technol Benevento Italy OHB Italia SpA Via Gallarate 150 I-20151 Milan Italy Univ Sannio Dept Engn Benevento Italy

ISBN: (纸本)9781509042340

Spacecraft on-board autonomy is an important topic in currently developed and future space missions. In this study, we present a robust approach to the optimal policy of autonomous space systems modeled via Markov Decision Process (MDP) from the values assigned to its transition probability matrix. After addressing the curse of dimensionality in solving the formulated MDP problem via approximate dynamic programming, we use an Apriori-based Association Classifier to infer a specific optimal policy. Finally, we also assess the effectiveness of such optimal policy in fulfilling the spacecraft autonomy requirements.

关键词： Spacecraft Autonomy Markov Decision Process approximate dynamic programming Machine Learning Association Classifier Apriori-based classifier

来源：评论

学校读者我要写书评

暂无评论

Relations between Model Predictive Control and Reinforcement Learning

Relations between Model Predictive Control and Reinforcement...

引用

20th World Congress of the International-Federation-of-Automatic-Control (IFAC)

作者： Goerges, Daniel Univ Kaiserslautern Electromobil Erwin Schrodinger Str 12 D-67663 Kaiserslautern Germany

In this paper relations between model predictive control and reinforcement learning are studied for discrete-time linear time-invariant systems with state and input constraints and a quadratic value function. The principles of model predictive control and reinforcement learning are reviewed in a tutorial manner. From model predictive control theory it is inferred that the optimal value function is piecewise quadratic on polyhedra and that the optimal policy is piecewise affine on polyhedra. Various ideas for exploiting the knowledge on the structure and the properties of the optimal value function and the optimal policy in reinforcement learning theory and practice are presented. The ideas can be used for deriving stability and feasibility criteria and for accelerating the learning process which can facilitate reinforcement learning for systems with high order, fast dynamics, and strict safety requirements. (C) 2017, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.

关键词： Model predictive control multi-parametric programming reinforcement learning approximate dynamic programming actor-critic structure

来源：评论

学校读者我要写书评

暂无评论

Four Nonlinear Multi-input Multi-output ADHDP Constructions and Algorithms Based on Topology Principle 4

Four Nonlinear Multi-input Multi-output ADHDP Constructions ...

引用

4th International Conference on Systems and Informatics (ICSAI)

作者： Huang, Zhijian Zhang, Cheng Zheng, Huan Wang, Shengtang Liu, Yihua Zhang, Guichen Huang, Xing Shanghai Maritime Univ Lab Intelligent Control & Computat Shanghai Peoples R China

ISBN: (纸本)9781538611074

In this paper, Four action-dependent heuristic dynamic programming control methods are presented for nonlinear multi-input-multi-output system with different characters based on the topology principle. These four methods are the action-network extension method, the sub-network method, the cascaded action-network method and the combined method. The derivation procedure and computing formulas of these methods are also derived. In it, the action-network extension method is mainly used for the conditions where the multi-output variables have the same orders of magnitude and a naturally coupled relationship. The sub-network method can nearly be applied in all cases and can solve the problem that the multi-output variables have different orders of magnitude. The cascaded action-network method is utilized when the multiple input variables have explicit cascaded relationships. The combined method can be used to control some highly regarded systems. Thus, these four methods can almost be used to satisfy all the design requirements of the nonlinear multi-input-multi-output control systems. The latter can refer to and select these methods as well as formulas for their control systems according to the research results to achieve a better control effect.

关键词： approximate dynamic programming nonlinear multi-input-multi-output method

来源：评论

学校读者我要写书评

暂无评论

Stochastic Zero-Sum Nash Games for Uncertain Nonlinear Markovian Jump Systems 56

Stochastic Zero-Sum Nash Games for Uncertain Nonlinear Marko...

引用

56th Annual IEEE Conference on Decision and Control (CDC)

作者： Vamvoudakis, Kyriakos G. Safaei, Farshad R. Pour Virginia Tech Dept Aerosp & Ocean Engn Blacksburg VA 24061 USA Bosch Res & Technol Ctr North Amer RTC Palo Alto CA 94304 USA

ISBN: (纸本)9781509028733

In this paper, a novel adaptive learning technique is proposed to solve a stochastic zero-sum Nash game with partially unknown nonlinear systems for which the lengths of time intervals that the system spends in each mode are independent random variables with exponential distributions, i.e. the environment and the cost matrices depend on the outcome of a Markov chain. We first formulate the problem by using an optimal stopping process and then provide a verification theorem for stopping zero-sum games. A structure of 2 actors and 1 critic approximators are used to approximate the saddle-point policies and the optimal cost respectively. Effective tuning laws are proposed to solve the stochastic Nash game problem while also guaranteeing closed-loop stability with the use of rigorous Lyapunov-based stability proofs. Finally, a numerical example is used to illustrate the effectiveness of the proposed approach.

关键词： Stochastic Nash game optimal stopping process approximate dynamic programming Markov jump systems

来源：评论

学校读者我要写书评

暂无评论

Active Fault Diagnosis for Jump Markov Nonlinear Systems

引用

IFAC-PapersOnLine 2017年第1期50卷 7308-7313页

作者： Škach J. Punčochář I. Straka O. European Centre of Excellence - NTIS Faculty of Applied Sciences University of West Bohemia Pilsen 306 14 Czech Republic

In this paper, a problem of active fault diagnosis for jump Markov nonlinear systems with non-Gaussian noises is considered. The imperfect state information formulation is transformed using sufficient statistics to a dynamical optimization problem that can be solved using approximate dynamic programming. The sufficient statistics are produced using the Bayesian recursive relations and particle filter algorithm. A special structure of approximate Bellman function is chosen to reduce a complexity caused by high dimension of statistics obtained from the particle filter. The proposed active fault detector design is compared with an extended Kalman filter based design in the simulation example. © 2017

关键词： Active fault diagnosis approximate dynamic programming jump Markov systems nonlinear state estimation optimization particle filter

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：