检索结果-内蒙古大学图书馆

Neural-Network-Based Robust Control Schemes for Nonlinear Multiplayer Systems With Uncertainties via Adaptive dynamic programming

引用

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2019年第3期49卷 579-588页

作者： Jiang, He Zhang, Huaguang Luo, Yanhong Han, Ji Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China

This paper investigates the robust control issues of nonlinear multiplayer systems by utilizing adaptive dynamic programming (ADP) methods and fills a gap in the ADP field, where actuator uncertainties for multiplayer systems are still not addressed. Two types of actuator uncertainties including bounded nonlinear perturbation and unknown constant actuator fault are taken into consideration. First, a data-driven reinforcement learning (RL) approach is derived to learn the optimal solutions of multiplayer nonzero-sum games. Then, based on the obtained optimal control policies, two robust control schemes are developed to handle these two different types of uncertainties, respectively, and the associated stability analysis is also provided. To implement the proposed iterative RL approach, a single neural network (NN) architecture with least-square-based updating law is given, which reduces the computation burden compared with the traditional dual NN architecture. Finally, two numerical examples are shown to test the feasibility of our proposed schemes.

关键词： Adaptive dynamic programming (ADP) approximate dynamic programming neural network (NN) reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Model-based reinforcement learning for nonlinear optimal control with practical asymptotic stability guarantees

引用

AICHE JOURNAL 2020年第10期66卷 e16544-e16544页

作者： Kim, Yeonsoo Lee, Jong Min Seoul Natl Univ Sch Chem & Biol Engn Inst Chem Proc 1 Gwanak Ro Seoul 08826 South Korea

We propose a new reinforcement learning approach for nonlinear optimal control where the value function is updated as restricted to control Lyapunov function (CLF) and the policy is improved using a variation of Sontag's formula. The practical asymptotic stability of the closed-loop system is guaranteed during the training and at the end of training without requiring an additional actor network and its update rule. For a single-layer neural network (NN) with exact basis functions, the approximate function converges to the optimal value function, resulting in the optimal controller. When a deep NN is used, the level set shapes of the trained NN become similar to those of the optimal value function. Because Sontag's formula with CLF is equivalent to the optimal controller when the given CLF has the same level set shapes as the optimal value function, Sontag's formula with the trained NN provides a nearly optimal controller.

关键词： approximate dynamic programming control Lyapunov function reinforcement learning Sontag's formula stability

来源：评论

学校读者我要写书评

暂无评论

Adaptive State Space Partitioning for dynamic Decision Processes

引用

BUSINESS & INFORMATION SYSTEMS ENGINEERING 2019年第3期61卷 261-275页

作者： Soeffker, Ninja Ulmer, Marlin W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig Muhlenpfordtstr 23 D-38106 Braunschweig Germany

With the rise of new business processes that require real-time decision making, anticipatory decision making becomes necessary to use the available resources wisely. dynamic real-time problems occur in many business fields, for example in vehicle routing applications with stochastic customer service requests expecting a fast response. For anticipatory decision making, offline simulation-based optimization methods like value function approximation are promising solution approaches. However, these methods require a suitable approximation architecture to store the value information for the problem states. In this paper, an approach is proposed that finds and adapts this architecture iteratively during the approximation process. A computational proof of concept is presented for a dynamic vehicle routing problem. In comparison to conventional architectures, the proposed method is able to improve the solution quality and reduces the required architecture size significantly.

关键词： approximate dynamic programming dynamic service routing State space partitioning Data-driven modeling and simulation Simulation-based optimization

来源：评论

学校读者我要写书评

暂无评论

Preemptive depot returns for dynamic same-day delivery

引用

EURO JOURNAL ON TRANSPORTATION AND LOGISTICS 2019年第4期8卷 327-361页

作者： Ulmer, Marlin W. Thomas, Barrett W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany Univ Iowa Iowa City IA USA

In this paper, we explore same-day delivery routing and particularly how same-day delivery vehicles can better integrate dynamic requests into delivery routes by taking advantage of preemptive depot returns. A preemptive depot return occurs when a delivery vehicle returns to the depot before delivering all of the packages currently on-board the vehicle. In this paper, we assume that a vehicle serves requests in a particular delivery area. Beginning the day with some known deliveries, the vehicle seeks to serve the known requests as well as additional new requests that are received throughout the day. To serve the new requests, the vehicle must return to the depot to pick up the packages for delivery. In contrast to previous work on same-day delivery routing, in this paper, we allow the vehicle to return to the depot before serving all loaded packages. To solve the problem, we couple an approximation of the value of choosing any particular subset of requests for delivery with a routing heuristic. Our approximation procedure is based on approximate dynamic programming and allows us to capture both the current value of a subset selection decision and its impact on future rewards. Using extensive computational tests, we demonstrate the value of preemptive depot returns and the value of the proposed approximation scheme in supporting preemptive returns. We also identify characteristics of instances for which preemptive depot returns are most likely to offer improvement.

关键词： Stochastic dynamic vehicle routing Same-day delivery Preemptive depot returns approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Advanced techniques of power system restoration and practical applications in transmission grids

引用

ELECTRIC POWER SYSTEMS RESEARCH 2020年第0期182卷 106238-000页

作者： Sharma, Dhruv Lin, Chenxi Luo, Xiaochuan Wu, Di Thulasiraman, Krishnaiya Jiang, John N. VIT Bhopal Univ Bhopal India Eleon Energy Austin TX USA ISO New England Holyoke MA USA North Dakota State Univ Fargo ND USA Univ Oklahoma Norman OK 73019 USA

Optimal restoration of a real-time power system following a disruption is a complex process. In view of that and with increase in frequency and severity of power system outages across the US and their impact on consumers and utilities, North American Electric Reliability Corporation elevated the standard of compliance for power system restoration. While several utilities have proposed solutions addressing the elevated standards based on dynamic programming, they could not address the issues for large-scale power systems and real-time operations including those using steady-state and transient analysis due to the curses of dimensionality. In this paper, we introduce a restoration process based on approximate dynamic programming integrated with solution space reduction methodology. The main theoretical foundations of the methodology and key algorithms used in this restoration process include the knowledge of power system engineering, complex network science, concepts of graph theory, and dynamic programming in operation research. We first use the proposed method to demonstrate an optimal restoration process on the IEEE 118-bus test system and then on a 2000-bus synthetic test system. This method addresses the concerns related to the curses of dimensionality and simplifies the solution space and thus can be applied to various complex real-time operation settings.

关键词： Power system restoration approximate dynamic programming Solution space reduction

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for dynamic capacity allocation with multiple priority levels

引用

IIE TRANSACTIONS 2011年第2期43卷 129-142页

作者： Erdelyi, Alexander Topaloglu, Huseyin Cornell Univ Sch Operat Res & Informat Engn Ithaca NY 14853 USA

This article considers a quite general dynamic capacity allocation problem. There is a fixed amount of daily processing capacity. On each day, jobs of different priorities arrive randomly and a decision has to made about which jobs should be scheduled on which days. Waiting jobs incur a holding cost that is a function of their priority levels. The objective is to minimize the total expected cost over a finite planning horizon. The problem is formulated as a dynamic program, but this formulation is computationally difficult as it involves a high-dimensional state vector. To address this difficulty, an approximate dynamic programming approach is used that decomposes the dynamic programming formulation by the different days in the planning horizon to construct separable approximations to the value functions. Value function approximations are used for two purposes. First, it is shown that the value function approximations can be used to obtain a lower bound on the optimal total expected cost. Second, the value function approximations can be used to make the job scheduling decisions over time. Computational experiments indicate that the job scheduling decisions made by the proposed approach perform significantly better than a variety of benchmark strategies.

关键词： approximate dynamic programming capacity allocation scheduling

来源：评论

学校读者我要写书评

暂无评论

Robust Control of Unknown Observable Nonlinear Systems Solved as a Zero-Sum Game

引用

IEEE ACCESS 2020年 8卷 214153-214165页

作者： Radac, Mircea-Bogdan Lala, Timotei Politehn Univ Timisoara Dept Automat & Appl Informat Timisoara 300223 Romania

An optimal robust control solution for general nonlinear systems with unknown but observable dynamics is advanced here. The underlying Hamilton-Jacobi-Isaacs (HJI) equation of the corresponding zero-sum two-player game (ZS-TP-G) is learned using a Q-learning-based approach employing only input-output system measurements, assuming system observability. An equivalent virtual state-space model is built from the system's input-output samples and it is shown that controlling the former implies controlling the latter. Since the existence of a saddle-point solution to the ZS-TP-G is assumed unverifiable, the solution is derived in terms of upper-optimal and lower-optimal controllers. The learning convergence is theoretically ensured while practical implementation is performed using neural networks that provide scalability to the control problem dimension and automatic feature selection. The learning strategy is checked on an active suspension system, a good candidate for the robust control problem with respect to road profile disturbance rejection.

关键词： Mathematical model Robust control Games Optimal control Linear systems Game theory Roads Active suspension system approximate dynamic programming neural networks optimal control reinforcement learning state feedback zero-sum two-player games

来源：评论

学校读者我要写书评

暂无评论

Output feedback adaptive dynamic programming for linear differential zero-sum games

引用

AUTOMATICA 2020年 122卷 109272-109272页

作者： Rizvi, Syed Ali Asad Lin, Zongli Brown, Charles L. Univ Virginia Charles L Brown Dept Elect & Comp Engn POB 400743 Charlottesville VA 22904 USA

This paper addresses the problem of finding optimal output feedback strategies for solving linear differential zero-sum games using a model-free approach based on adaptive dynamic programming (ADP). In contrast to their discrete-time counterparts, differential games involve continuous-time dynamics and existing ADP approaches to their solutions require access to full measurement of the internal state. This difficulty is due to the fact that direct translation of the discrete-time output feedback ADP results requires derivatives of the input and output measurements, which is generally prohibitive in practice. This work aims to overcome this difficulty and presents a new embedded filtering based observer approach towards designing output feedback ADP algorithms for solving the differential zero-sum game problem. Two output feedback ADP algorithms based respectively on policy iteration and value iteration are developed. The proposed scheme is completely online in nature and works without requiring information of the system dynamics. In addition, this work also addresses the excitation bias problem encountered in output feedback ADP methods, which typically requires a discounting factor for its mitigation. We show that the proposed scheme is bias-free, and therefore, does not require a discounting factor. It is shown that the proposed algorithms converge to the solution obtained by solving the game algebraic Riccati equation. Two numerical examples are demonstrated to validate the proposed scheme. (c) 2020 Elsevier Ltd. All rights reserved.

关键词： Adaptive dynamic programming approximate dynamic programming Zero-sum games Output feedback

来源：评论

学校读者我要写书评

暂无评论

Train trajectory optimization for improved on-time arrival under parametric uncertainty

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2020年 119卷 102680-102680页

作者： Wang, Pengling Trivella, Alessio Goverde, Rob M. P. Corman, Francesco Swiss Fed Inst Technol IVT Inst Transport Planning & Syst Zurich Switzerland Delft Univ Technol Dept Transport & Planning Delft Netherlands

In this paper we study the problem of computing train trajectories in an uncertain environment in which the values of some system parameters are difficult to determine. Specifically, we consider uncertainty in traction force and train resistance, and their impact on travel time and energy consumption. Our ultimate goal is to be able to control trains such that they will arrive on-time, i.e. within the planned running time, regardless of uncertain factors affecting their dynamic or kinematic performance. We formulate the problem as a Markov decision process and solve it using a novel numerical approach which combines: (i) an off-line approximate dynamic programming (ADP) method to learn the energy and time costs over iterations, and (ii) an on-line search process to determine energy-efficient driving strategies that respect the real-time time windows, more in general expressed as train path envelope constraints. To evaluate the performance of our approach, we conducted a numerical study using real-life railway infrastructure and train data. Compared to a set of benchmark driving strategies, the trajectories from our ADP-based method reduce the probability of delayed arrival, and at the same time are able to better use the available running time for energy saving. Our results show that accounting for uncertainty is relevant when computing train trajectories and that our ADP-based method can handle this uncertainty effectively.

关键词： Train trajectory optimization Parametric uncertainty approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for an inventory problem: Empirical comparison

引用

COMPUTERS & INDUSTRIAL ENGINEERING 2011年第4期60卷 719-743页

作者： Katanyukul, Tatpong Duff, William S. Chong, Edwin K. P. Colorado State Univ Coll Engn Dept Mech Engn Ft Collins CO 80523 USA Colorado State Univ Coll Engn Elect & Comp Engn Dept Ft Collins CO 80523 USA

This study investigates the application of learning-based and simulation-based approximate dynamic programming (ADP) approaches to an inventory problem under the Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model. Specifically, we explore the robustness of a learning-based ADP method, Sarsa, with a GARCH(1,1) demand model, and provide empirical comparison between Sarsa and two simulation-based ADP methods: Rollout and Hindsight Optimization (HO). Our findings assuage a concern regarding the effect of GARCH(1,1) latent state variables on learning-based ADP and provide practical strategies to design an appropriate ADP method for inventory problems. In addition, we expose a relationship between ADP parameters and conservative behavior. Our empirical results are based on a variety of problem settings, including demand correlations, demand variances, and cost structures. (C) 2011 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Inventory control Reinforcement learning Simulation Heterogeneity AR(1)/GARCH(1,1)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：