检索结果-内蒙古大学图书馆

The State Following Approximation Method

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019年第6期30卷 1716-1730页

作者： Rosenfeld, Joel A. Kamalapurkar, Rushikesh Dixon, Warren E. Vanderbilt Univ Dept Elect Engn & Comp Sci 221 Kirkland Hall Nashville TN 37235 USA Oklahoma State Univ Dept Mech & Aerosp Engn Syst Cognit & Control Lab Stillwater OK 74078 USA Univ Florida Dept Mech & Aerosp Engn Nonlinear Controls & Robot Lab Gainesville FL 32611 USA

A function approximation method is developed which aims to approximate a function in a small neighborhood of a state that travels within a compact set. The method provides a novel approximation strategy for the efficient approximation of nonlinear functions for real-time simulations and experiments. The development is based on the theory of universal reproducing kernel Hilbert spaces over the n-dimensional Euclidean space. Several theorems are introduced which support the development of this state following (StaF) method. In particular, it is shown that there is a bound on the number of kernel functions required for the maintenance of an accurate function approximation as a state moves through a compact set. In addition, a weight update law, based on gradient descent, is introduced where arbitrarily close accuracy can be achieved provided the weight update law is iterated at a sufficient frequency, as detailed in Theorem 4. An experience-based approximation method is presented which utilizes the samples of the estimations of the ideal weights to generate a global approximation of a function. The experience-based approximation interpolates the samples of the weight estimates using radial basis functions. To illustrate the StaF method, the method is utilized for derivative estimation, function approximation, and is applied to an adaptive dynamic programming problem where it is demonstrated that the stability is maintained with a reduced number of basis functions.

关键词： approximate dynamic programming approximation theory reproducing kernel Hilbert spaces state following approximation

来源：评论

学校读者我要写书评

暂无评论

Model-based reinforcement learning for nonlinear optimal control with practical asymptotic stability guarantees

引用

AICHE JOURNAL 2020年第10期66卷

作者： Kim, Yeonsoo Lee, Jong Min Seoul Natl Univ Sch Chem & Biol Engn Inst Chem Proc 1 Gwanak Ro Seoul 08826 South Korea

We propose a new reinforcement learning approach for nonlinear optimal control where the value function is updated as restricted to control Lyapunov function (CLF) and the policy is improved using a variation of Sontag's formula. The practical asymptotic stability of the closed-loop system is guaranteed during the training and at the end of training without requiring an additional actor network and its update rule. For a single-layer neural network (NN) with exact basis functions, the approximate function converges to the optimal value function, resulting in the optimal controller. When a deep NN is used, the level set shapes of the trained NN become similar to those of the optimal value function. Because Sontag's formula with CLF is equivalent to the optimal controller when the given CLF has the same level set shapes as the optimal value function, Sontag's formula with the trained NN provides a nearly optimal controller.

关键词： approximate dynamic programming control Lyapunov function reinforcement learning Sontag's formula stability

来源：评论

学校读者我要写书评

暂无评论

Output feedback adaptive dynamic programming for linear differential zero-sum games

引用

AUTOMATICA 2020年 122卷

作者： Rizvi, Syed Ali Asad Lin, Zongli Brown, Charles L. Univ Virginia Charles L Brown Dept Elect & Comp Engn POB 400743 Charlottesville VA 22904 USA

This paper addresses the problem of finding optimal output feedback strategies for solving linear differential zero-sum games using a model-free approach based on adaptive dynamic programming (ADP). In contrast to their discrete-time counterparts, differential games involve continuous-time dynamics and existing ADP approaches to their solutions require access to full measurement of the internal state. This difficulty is due to the fact that direct translation of the discrete-time output feedback ADP results requires derivatives of the input and output measurements, which is generally prohibitive in practice. This work aims to overcome this difficulty and presents a new embedded filtering based observer approach towards designing output feedback ADP algorithms for solving the differential zero-sum game problem. Two output feedback ADP algorithms based respectively on policy iteration and value iteration are developed. The proposed scheme is completely online in nature and works without requiring information of the system dynamics. In addition, this work also addresses the excitation bias problem encountered in output feedback ADP methods, which typically requires a discounting factor for its mitigation. We show that the proposed scheme is bias-free, and therefore, does not require a discounting factor. It is shown that the proposed algorithms converge to the solution obtained by solving the game algebraic Riccati equation. Two numerical examples are demonstrated to validate the proposed scheme. (c) 2020 Elsevier Ltd. All rights reserved.

关键词： Adaptive dynamic programming approximate dynamic programming Zero-sum games Output feedback

来源：评论

学校读者我要写书评

暂无评论

Adaptive State Space Partitioning for dynamic Decision Processes

引用

BUSINESS & INFORMATION SYSTEMS ENGINEERING 2019年第3期61卷 261-275页

作者： Soeffker, Ninja Ulmer, Marlin W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig Muhlenpfordtstr 23 D-38106 Braunschweig Germany

With the rise of new business processes that require real-time decision making, anticipatory decision making becomes necessary to use the available resources wisely. dynamic real-time problems occur in many business fields, for example in vehicle routing applications with stochastic customer service requests expecting a fast response. For anticipatory decision making, offline simulation-based optimization methods like value function approximation are promising solution approaches. However, these methods require a suitable approximation architecture to store the value information for the problem states. In this paper, an approach is proposed that finds and adapts this architecture iteratively during the approximation process. A computational proof of concept is presented for a dynamic vehicle routing problem. In comparison to conventional architectures, the proposed method is able to improve the solution quality and reduces the required architecture size significantly.

关键词： approximate dynamic programming dynamic service routing State space partitioning Data-driven modeling and simulation Simulation-based optimization

来源：评论

学校读者我要写书评

暂无评论

Preemptive depot returns for dynamic same-day delivery

引用

EURO JOURNAL ON TRANSPORTATION AND LOGISTICS 2019年第4期8卷 327-361页

作者： Ulmer, Marlin W. Thomas, Barrett W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany Univ Iowa Iowa City IA USA

In this paper, we explore same-day delivery routing and particularly how same-day delivery vehicles can better integrate dynamic requests into delivery routes by taking advantage of preemptive depot returns. A preemptive depot return occurs when a delivery vehicle returns to the depot before delivering all of the packages currently on-board the vehicle. In this paper, we assume that a vehicle serves requests in a particular delivery area. Beginning the day with some known deliveries, the vehicle seeks to serve the known requests as well as additional new requests that are received throughout the day. To serve the new requests, the vehicle must return to the depot to pick up the packages for delivery. In contrast to previous work on same-day delivery routing, in this paper, we allow the vehicle to return to the depot before serving all loaded packages. To solve the problem, we couple an approximation of the value of choosing any particular subset of requests for delivery with a routing heuristic. Our approximation procedure is based on approximate dynamic programming and allows us to capture both the current value of a subset selection decision and its impact on future rewards. Using extensive computational tests, we demonstrate the value of preemptive depot returns and the value of the proposed approximation scheme in supporting preemptive returns. We also identify characteristics of instances for which preemptive depot returns are most likely to offer improvement.

关键词： Stochastic dynamic vehicle routing Same-day delivery Preemptive depot returns approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

dynamic Incentive Mechanism for Delivery Slot Management in E-Commerce Attended Home Delivery

引用

TRANSPORTATION SCIENCE 2020年第3期54卷 567-587页

作者： Vinsensius, Albert Wang, Yuan Chew, Ek Peng Lee, Loo Hay Natl Univ Singapore Dept Ind Syst Engn & Management Singapore 117576 Singapore

In attended home delivery, challenges arise in practice because of the short strict time windows, stochastic customer requests, and varying customers' preferences for delivery slots. In this study, we focus on integrating dynamic time slot incentives and order delivery with the intention of reducing overall delivery cost and improving profitability. The proposed incentive mechanism is able to exploit the variability in the marginal fulfillment cost of an order and the customers' preferences to influence the customers' selection of delivery slots. We present an approximate dynamic programming approach to estimate the marginal fulfillment cost using the operational vehicle routing cost while accounting for future orders. We demonstrate that the proposed incentive mechanism can achieve a high level of savings (of up to 70%) with respect to the benchmark customer-free-choice scenario. It is also noted that the proposed mechanism effectively exploits higher order density and vehicle availability to achieve a higher level of savings.

关键词： dynamic incentives attended home delivery approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

On the Taylor Expansion of Value Functions

引用

OPERATIONS RESEARCH 2020年第2期68卷 631-654页

作者： Braverman, Anton Gurvich, Itai Huang, Junfei Northwestern Univ Kellogg Sch Management Evanston IL 60208 USA Cornell Sch Operat Res & Informat Engn New York NY 10044 USA Cornell Tech New York NY 10044 USA Chinese Univ Hong Kong CUHK Business Sch Dept Decis Sci & Managerial Econ Shatin Hong Kong Peoples R China

We introduce a framework for approximate dynamic programming that we apply to discrete-time chains on Z(+)(d) with countable action sets. The framework is grounded in the approximation of the (controlled) chain's generator by that of another Markov process. In simple terms, our approach stipulates applying a second-order Taylor expansion to the value function, replacing the Bellman equation with one in continuous space and time in which the transition matrix is reduced to its first and second moments. In some cases, the resulting equation can be interpreted as a Hamilton-Jacobi-Bellman equation for a Brownian control problem. When tractable, the "Taylored" equation serves as a useful modeling tool. More generally, it is a starting point for approximation algorithms. We develop bounds on the optimality gap-the suboptimality introduced by using the control produced by the Taylored equation. These bounds can be viewed as a conceptual underpinning, analytical rather than relying on weak convergence arguments, for the good performance of controls derived from Brownian approximations. We prove that under suitable conditions and for suitably "large" initial states, (1) the optimality gap is smaller than a 1-alpha fraction of the optimal value, with which alpha is an element of (0, 1) is the discount factor, and (2) the gap can be further expressed as the infinite-horizon discounted value with a "lowerorder" per-period reward. Computationally, our framework leads to an "aggregation" approach with performance guarantees. Although the guarantees are grounded in partial differential equation theory, the practical use of this approach requires no knowledge of that theory.

关键词： Taylor expansion Markov decision process Hamilton-Jacobi-Bellman equation approximate dynamic programming approximate policy iteration

来源：评论

学校读者我要写书评

暂无评论

Advanced techniques of power system restoration and practical applications in transmission grids

引用

ELECTRIC POWER SYSTEMS RESEARCH 2020年 182卷

作者： Sharma, Dhruv Lin, Chenxi Luo, Xiaochuan Wu, Di Thulasiraman, Krishnaiya Jiang, John N. VIT Bhopal Univ Bhopal India Eleon Energy Austin TX USA ISO New England Holyoke MA USA North Dakota State Univ Fargo ND USA Univ Oklahoma Norman OK 73019 USA

Optimal restoration of a real-time power system following a disruption is a complex process. In view of that and with increase in frequency and severity of power system outages across the US and their impact on consumers and utilities, North American Electric Reliability Corporation elevated the standard of compliance for power system restoration. While several utilities have proposed solutions addressing the elevated standards based on dynamic programming, they could not address the issues for large-scale power systems and real-time operations including those using steady-state and transient analysis due to the curses of dimensionality. In this paper, we introduce a restoration process based on approximate dynamic programming integrated with solution space reduction methodology. The main theoretical foundations of the methodology and key algorithms used in this restoration process include the knowledge of power system engineering, complex network science, concepts of graph theory, and dynamic programming in operation research. We first use the proposed method to demonstrate an optimal restoration process on the IEEE 118-bus test system and then on a 2000-bus synthetic test system. This method addresses the concerns related to the curses of dimensionality and simplifies the solution space and thus can be applied to various complex real-time operation settings.

关键词： Power system restoration approximate dynamic programming Solution space reduction

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming for an inventory problem: Empirical comparison

引用

COMPUTERS & INDUSTRIAL ENGINEERING 2011年第4期60卷 719-743页

作者： Katanyukul, Tatpong Duff, William S. Chong, Edwin K. P. Colorado State Univ Coll Engn Dept Mech Engn Ft Collins CO 80523 USA Colorado State Univ Coll Engn Elect & Comp Engn Dept Ft Collins CO 80523 USA

This study investigates the application of learning-based and simulation-based approximate dynamic programming (ADP) approaches to an inventory problem under the Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model. Specifically, we explore the robustness of a learning-based ADP method, Sarsa, with a GARCH(1,1) demand model, and provide empirical comparison between Sarsa and two simulation-based ADP methods: Rollout and Hindsight Optimization (HO). Our findings assuage a concern regarding the effect of GARCH(1,1) latent state variables on learning-based ADP and provide practical strategies to design an appropriate ADP method for inventory problems. In addition, we expose a relationship between ADP parameters and conservative behavior. Our empirical results are based on a variety of problem settings, including demand correlations, demand variances, and cost structures. (C) 2011 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Inventory control Reinforcement learning Simulation Heterogeneity AR(1)/GARCH(1,1)

来源：评论

学校读者我要写书评

暂无评论

Robust Control of Unknown Observable Nonlinear Systems Solved as a Zero-Sum Game

引用

IEEE ACCESS 2020年 8卷 214153-214165页

作者： Radac, Mircea-Bogdan Lala, Timotei Politehn Univ Timisoara Dept Automat & Appl Informat Timisoara 300223 Romania

An optimal robust control solution for general nonlinear systems with unknown but observable dynamics is advanced here. The underlying Hamilton-Jacobi-Isaacs (HJI) equation of the corresponding zero-sum two-player game (ZS-TP-G) is learned using a Q-learning-based approach employing only input-output system measurements, assuming system observability. An equivalent virtual state-space model is built from the system's input-output samples and it is shown that controlling the former implies controlling the latter. Since the existence of a saddle-point solution to the ZS-TP-G is assumed unverifiable, the solution is derived in terms of upper-optimal and lower-optimal controllers. The learning convergence is theoretically ensured while practical implementation is performed using neural networks that provide scalability to the control problem dimension and automatic feature selection. The learning strategy is checked on an active suspension system, a good candidate for the robust control problem with respect to road profile disturbance rejection.

关键词： Mathematical model Robust control Games Optimal control Linear systems Game theory Roads Active suspension system approximate dynamic programming neural networks optimal control reinforcement learning state feedback zero-sum two-player games

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：