检索结果-内蒙古大学图书馆

A Comparison of approximate dynamic programming and Simple Genetic Algorithm for Traffic Control in Oversaturated Conditions - Case study of a Simple Symmetric Network

A Comparison of Approximate Dynamic Programming and Simple G...

引用

14th International IEEE Conference on Intelligent Transportation Systems (ITSC)

作者： Medina, Juan C. Hajbabaie, Ali Benekohal, Rahim F. Univ Illinois Urbana IL 61801 USA

ISBN: (纸本)9781457721977

The performance of two algorithms for finding traffic signal timings in a small symmetric network with oversaturated conditions was analyzed. The two algorithms include an approximate dynamic programming approach using a "post-decision" state variable (ADP) and a simple genetic algorithm (GA). Results were found by using microscopic simulation and compared based on typical measures of performance (delay, throughput, number of stops) and also on measures that considered the efficiency of green time utilization and queue occupancy of the links. The symmetric characteristics of the small network allowed a straightforward analysis of the operation of the signals, providing some insights on the quality of the solutions. Results showed that even though the solutions from ADP were very different from those in GA, the network performance for both methods was similar, used green time efficiently preventing queue backups, and served all approaches according to current demands. The potential of ADP using the "post-decision" state variable is currently under further analysis using more challenging conditions, additional constraints, and domain knowledge as part of the algorithm formulation.

关键词： Delay Equations Genetic algorithms Mathematical model Traffic control Vehicles approximate dynamic programming dynamic programming genetic algorithms green time utilization microscopic simulation oversaturated conditions post-decision state variable queue

来源：评论

学校读者我要写书评

暂无评论

Chaotic system optimal tracking using data-based synchronous method with unknown dynamics and disturbances

引用

Chinese Physics B 2017年第3期26卷 268-275页

作者：宋睿卓魏庆来 School of Automation and Electrical Engineering University of Science and Technology Beijing The State Key Laboratory of Management and Control for Complex Systems Institute of AutomationChinese Academy of Sciences

We develop an optimal tracking control method for chaotic system with unknown dynamics and disturbances. The method allows the optimal cost function and the corresponding tracking control to update synchronously. According to the tracking error and the reference dynamics, the augmented system is constructed. Then the optimal tracking control problem is defined. The policy iteration （PI） is introduced to solve the rain-max optimization problem. The off-policy adaptive dynamic programming （ADP） algorithm is then proposed to find the solution of the tracking Hamilton-Jacobi- Isaacs （HJI） equation online only using measured data and without any knowledge about the system dynamics. Critic neural network （CNN）, action neural network （ANN）, and disturbance neural network （DNN） are used to approximate the cost function, control, and disturbance. The weights of these networks compose the augmented weight matrix, and the uniformly ultimately bounded （UUB） of which is proven. The convergence of the tracking error system is also proven. Two examples are given to show the effectiveness of the proposed synchronous solution method for the chaotic system tracking problem.

关键词： adaptive dynamic programming approximate dynamic programming chaotic system zero-sum

来源：评论

学校读者我要写书评

暂无评论

Adaptive dynamic programming-Based Optimal Control Scheme for Energy Storage Systems With Solar Renewable Energy

引用

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS 2017年第7期64卷 5468-5478页

作者： Wei, Qinglai Shi, Guang Song, Ruizhuo Liu, Yu Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Chinese Acad Sci Inst Automat Beijing 100190 Peoples R China

In this paper, a novel optimal energy storage control scheme is investigated in smart grid environments with solar renewable energy. Based on the idea of adaptive dynamic programming (ADP), a self-learning algorithm is constructed to obtain the iterative control law sequence of the battery. Based on the data of the real-time electricity price (electricity rate in brief), the load demand (load in brief), and the solar renewable energy (solar energy in brief), the optimal performance index function, which minimizes the total electricity cost and simultaneously extends the battery's lifetime, is established. A new analysis method of the iterative ADP algorithm is developed to guarantee the convergence of the iterative value function to the optimum under iterative control law sequence for any time index in a period. Numerical results and comparisons are presented to illustrate the effectiveness of the developed algorithm.

关键词： Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming energy storage energy storage system optimal control solar renewable energy

来源：评论

学校读者我要写书评

暂无评论

Least squares approximate policy iteration for learning bid prices in choice-based revenue management

引用

COMPUTERS & OPERATIONS RESEARCH 2017年第0期77卷 240-253页

作者： Koch, Sebastian Univ Augsburg Chair Analyt & Optimizat Univ Str 16 D-86159 Augsburg Germany

We consider the revenue management problem of capacity control under customer choice behavior. An exact solution of the underlying stochastic dynamic program is difficult because of the multi-dimensional state space and, thus, approximate dynamic programming (ADP) techniques are widely used. The key idea of ADP is to encode the multi-dimensional state space by a small number of basis functions, often leading to a parametric approximation of the dynamic program's value function. In general, two classes of ADP techniques for learning value function approximations exist: mathematical programming and simulation. So far, the literature on capacity control largely focuses on the first class. In this paper, we develop a least squares approximate policy iteration (API) approach which belongs to the second class. Thereby, we suggest value function approximations that are linear in the parameters, and we estimate the parameters via linear least squares regression. Exploiting both exact and heuristic knowledge from the value function, we enforce structural constraints on the parameters to facilitate learning a good policy. We perform an extensive simulation study to investigate the performance of our approach. The results show that it is able to obtain competitive revenues compared to and often out-performs state-of-the-art capacity control methods in reasonable computational time. Depending on the scarcity of capacity and the point in time, revenue improvements of around 1% or more can be observed. Furthermore, the proposed approach contributes to simulation-based ADP, bringing forth research on numerically estimating piecewise linear value function approximations and their application in revenue management environments. (C) 2016 Elsevier Ltd. All rights reserved.

关键词： Revenue management Capacity control approximate dynamic programming approximate policy iteration

来源：评论

学校读者我要写书评

暂无评论

Optimal control for discrete-time systems with actuator saturation

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2017年第6期38卷 1071-1080页

作者： Lin, Qiao Wei, Qinglai Zhao, Bo Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

In this study, we use generalized policy iteration approximate dynamic programming (ADP) algorithm to design an optimal controller for a class of discrete-time systems with actuator saturation. A integral function is proposed to manage the saturation nonlinearity in actuators and then the generalized policy iteration ADP algorithm is developed to deal with the optimal control problem. Compared with other algorithm, the developed ADP algorithm includes 2 iteration procedures. In the present control scheme, 2 neural networks are introduced to approximate the control law and performance index function. Furthermore, numerical simulations illustrate the convergence and feasibility of the developed method.

关键词： approximate dynamic programming discrete time generalized policy iteration optimal control saturating actuators

来源：评论

学校读者我要写书评

暂无评论

Simulation-based decision support framework for dynamic ambulance redeployment in Singapore

引用

INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 2017年 106卷 37-47页

作者： Lam, Sean Shao Wei Ng, Clarence Boon Liang Nguyen, Francis Ngoc Hoang Long Ng, Yih Yng Ong, Marcus Eng Hock Singapore Hlth Serv Hlth Serv Res Ctr 20 Coll RdAcad Discovery TowerLevel 6 Singapore S169856 Singapore Duke NUS Grad Med Sch Hlth Serv & Syst Res Singapore Singapore Natl Univ Singapore Dept Ind & Syst Engn Singapore Singapore Singapore Civil Def Force Med Dept Singapore Singapore Singapore Gen Hosp Dept Emergency Med Singapore Singapore

Objective: dynamic ambulance redeployment policies tend to introduce much more flexibilities in improving ambulance resource allocation by capitalizing on the definite geospatial-temporal variations in ambulance demand patterns over the time-of-the-day and day-of-the-week effects. A novel modelling framework based on the approximate dynamic programming (ADP) approach leveraging on a Discrete Events Simulation (DES) model for dynamic ambulance redeployment in Singapore is proposed in this paper. Methods: The study was based on the Singapore's national Emergency Medical Services (EMS) system. Based on a dataset comprising 216,973 valid incidents over a continuous two-years study period from 1 January 2011-31 December 2012, a DES model for the EMS system was developed. An ADP model based on linear value function approximations was then evaluated using the DES model via the temporal difference (TD) learning family of algorithms. The objective of the ADP model is to derive approximate optimal dynamic redeployment policies based on the primary outcome of ambulance coverage. Results: Considering an 8 min response time threshold, an estimated 5% reduction in the proportion of calls that cannot be reached within the threshold (equivalent to approximately 8000 dispatches) was observed from the computational experiments. The study also revealed that the redeployment policies which are restricted within the same operational division could potentially result in a more promising response time performance. Furthermore, the best policy involved the combination of redeploying ambulances whenever they are released from service and that of relocating ambulances that are idle in bases. Conclusion: This study demonstrated the successful application of an approximate modelling framework based on ADP that leverages upon a detailed DES model of the Singapore's EMS system to generate approximate optimal dynamic redeployment plans. Various policies and scenarios relevant to the Singapore EMS

关键词： Emergency medical services Ambulance deployment approximate dynamic programming dynamic redeployment policies Discrete events simulation

来源：评论

学校读者我要写书评

暂无评论

Off-policy neuro-optimal control for unknown complex-valued nonlinear systems based on policy iteration

引用

NEURAL COMPUTING & APPLICATIONS 2017年第6期28卷 1435-1441页

作者： Song, Ruizhuo Wei, Qinglai Xiao, Wendong Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

This paper establishes an optimal control of unknown complex-valued system. Policy iteration is used to obtain the solution of the Hamilton-Jacobi-Bellman equation. Off-policy learning allows the iterative performance index and iterative control to be obtained by completely unknown dynamics. Critic and action networks are used to get the iterative control and iterative performance index, which execute policy evaluation and policy improvement. Asymptotic stability of the closed-loop system and the convergence of the iterative performance index function are proven. By Lyapunov technique, the uniformly ultimately bounded of the weight error is proven. Simulation study demonstrates the effectiveness of the proposed optimal control method.

关键词： Adaptive dynamic programming approximate dynamic programming Adaptive critic designs Optimal control

来源：评论

学校读者我要写书评

暂无评论

Sensitivity-based nested partitions for solving finite-horizon Markov decision processes

引用

OPERATIONS RESEARCH LETTERS 2017年第5期45卷 481-487页

作者： Chen, Weiwei Rutgers State Univ Dept Supply Chain Management 1 Washington Pk Newark NJ 07102 USA

In this paper, we propose a heuristic for solving finite-horizon Markov decision processes. The heuristic uses the nested partitions (NP) framework to guide an iterative search for the optimal policy. NP focuses the search on certain promising subregions, flexibly determined by the sampling weight of each action branch. Within each subregion, an effective local policy optimization is developed using sensitivity-based approach, which optimizes the sampling weights based on estimated gradient information. Numerical results show the effectiveness of the proposed heuristic. (C) 2017 Elsevier B.V. All rights reserved.

关键词： approximate dynamic programming Markov decision processes Nested partitions Sensitivity-based approach

来源：评论

学校读者我要写书评

暂无评论

Spatial interactions and optimal forest management on a fire-threatened landscape

引用

FOREST POLICY AND ECONOMICS 2017年 83卷 107-120页

作者： Lauer, Christopher J. Montgomery, Claire A. Dietterich, Thomas G. Oregon State Univ Appl Econ 213 Ballard Extens Hall Corvallis OR 97331 USA Oregon State Univ Forest Engn Resources &Management Corvallis OR 97331 USA Oregon State Univ Elect Engn & Comp Sci 2067 Kelly Engn Ctr Corvallis OR 97331 USA

Forest management in the face of fire risk is a challenging problem because fire spreads across a landscape and because its occurrence is unpredictable. Accounting for the existence of stochastic events that generate spatial interactions in the context of a dynamic decision process is crucial for determining optimal management. This paper demonstrates a method for incorporating spatial information and interactions into management decisions made over time. A machine learning technique called approximate dynamic programming is applied to determine the optimal timing and location of fuel treatments and timber harvests for a fire-threatened landscape. Larger net present values can be achieved using policies that explicitly consider evolving spatial interactions created by fire spread, compared to policies that ignore the spatial dimension of the inter-temporal optimization problem.

关键词： Wildland fire Spatial Ecological disturbance Risk approximate dynamic programming Reinforcement learning Forestry

来源：评论

学校读者我要写书评

暂无评论

Discrete-Time Deterministic Q-Learning: A Novel Convergence Analysis

引用

IEEE TRANSACTIONS ON CYBERNETICS 2017年第5期47卷 1224-1237页

作者： Wei, Qinglai Lewis, Frank L. Sun, Qiuye Yan, Pengfei Song, Ruizhuo Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Texas Arlington UTA Res Inst Arlington TX 76118 USA Northeastern Univ Shenyang 110036 Peoples R China Northeastern Univ Sch Informat Sci & Engn Shenyang 110036 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China

In this paper, a novel discrete-time deterministic Q-learning algorithm is developed. In each iteration of the developed Q-learning algorithm, the iterative Q function is updated for all the state and control spaces, instead of updating for a single state and a single control in traditional Q-learning algorithm. A new convergence criterion is established to guarantee that the iterative Q function converges to the optimum, where the convergence criterion of the learning rates for traditional Q-learning algorithms is simplified. During the convergence analysis, the upper and lower bounds of the iterative Q function are analyzed to obtain the convergence criterion, instead of analyzing the iterative Q function itself. For convenience of analysis, the convergence properties for undiscounted case of the deterministic Q-learning algorithm are first developed. Then, considering the discounted factor, the convergence criterion for the discounted case is established. Neural networks are used to approximate the iterative Q function and compute the iterative control law, respectively, for facilitating the implementation of the deterministic Q-learning algorithm. Finally, simulation results and comparisons are given to illustrate the performance of the developed algorithm.

关键词： Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming neural networks (NNs) neuro-dynamic programming optimal control Q-learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：