检索结果-内蒙古大学图书馆

MODEL-FREE MULTI-KERNEL LEARNING CONTROL FOR NONLINEAR DISCRETE-TIME SYSTEMS

INTERNATIONAL JOURNAL OF ROBOTICS & AUTOMATION 2017年第5期32卷 538-550页

作者： Liu, Jiahang Xu, Xin Huang, Zhenhua Lian, Chuanqiang Natl Univ Def Technol Coll Mechatron & Automat Changsha Hunan Peoples R China Naval Univ Engn Wuhan Hubei Peoples R China

Reinforcement learning (RL) has become an important research topic to solve learning control problems of nonlinear dynamic systems. In RL, feature representation is a critical factor for improving the performance of online or offline learning controllers. Although multi-kernel learning has been studied in supervised learning problems, there is little work on multi-kernel-based feature representation in RL algorithms. In this paper, a model-free multi-kernel learning control (MMLC) approach is proposed for a class of nonlinear discrete-time systems. MMLC has advantages over other single-kernel-based RL algorithms in that the parameters in the kernel functions can be learned adaptively. Furthermore, MMLC uses a model-free actor-critic learning structure, where the critic is designed to approximate the derivatives of value functions. Different from the popularly studied dual heuristical programming algorithm, the proposed MMLC approach can learn the dynamics of the nonlinear system in a data-driven way. To evaluate the performance of MMLC, single-link and double-link inverted pendulums are employed as two benchmarks. The effectiveness of the MMLC algorithm has been demonstrated in simulation. It is shown that MMLC can achieve better performance than previous kernel-based dual heuristic programming with partial model information.

关键词： Reinforcement learning multiple kernel learning approximate dynamic programming model-free nonlinear systems

来源：评论

学校读者我要写书评

暂无评论

Tutorial on Stochastic Optimization in Energy-Part I: Modeling and Policies

引用

IEEE TRANSACTIONS ON POWER SYSTEMS 2016年第2期31卷 1459-1467页

作者： Powell, Warren B. Meisel, Stephan Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08544 USA Univ Munster Dept Informat Syst Munster Germany

There is a wide range of problems in energy systems that require making decisions in the presence of different forms of uncertainty. The fields that address sequential, stochastic decision problems lack a standard canonical modeling framework, with fragmented, competing solution strategies. Recognizing that we will never agree on a single notational system, this two-part tutorial proposes a simple, straightforward canonical model (that is most familiar to people with a control theory background), and introduces four fundamental classes of policies which integrate the competing strategies that have been proposed under names such as control theory, dynamic programming, stochastic programming and robust optimization. Part II of the tutorial illustrates the modeling framework using a simple energy storage problem, where we show that, depending on the problem characteristics, each of the four classes of policies may be best.

关键词： approximate dynamic programming dynamic programming energy systems optimal control reinforcement learning robust optimization stochastic optimization stochastic programming

来源：评论

学校读者我要写书评

暂无评论

Deep reinforcement learning based finite-horizon optimal control for a discrete-time affine nonlinear system

Deep reinforcement learning based finite-horizon optimal con...

引用

SICE Annual Conference

作者： Jong Woo Kim Byung Jun Park Haeun Yoo Jay H. Lee Jong Min Lee School of Chemical and Biological Engineering Seoul National University Seoul Republic of Korea Chemical and Biomolecular Engineering Department Korea Advanced Institute of Science and Technology Daejeon Republic of Korea

approximate dynamic programming (ADP) aims to obtain an approximate numerical solution to the discrete-time Hamilton-Jacobi-Bellman (HJB) equation. Heuristic dynamic programming (HDP) is a two-stage iterative scheme of ADP by separating the HJB equation into two equations, one for the value function and another for the policy function, which are referred to as the critic and the actor, respectively. Previous ADP implementations have been limited by the choice of function approximator, which requires significant prior domain knowledge or a large number of parameters to be fitted. However, recent advances in deep learning brought by the computer science community enable the use of deep neural networks (DNN) to approximate high-dimensional nonlinear functions without prior domain knowledge. Motivated by this, we examine the potential of DNNs as function approximators of the critic and the actor. In contrast to the infinite-horizon optimal control problem, the critic and the actor of the finite horizon optimal control (FHOC) problem are time-varying functions and have to satisfy a boundary condition. DNN structure and training algorithm suitable for FHOC are presented. Illustrative examples are provided to demonstrate the validity of the proposed method.

关键词： Reinforcement learning approximate dynamic programming Deep learning Actor-critic method Finite horizon optimal control

来源：评论

学校读者我要写书评

暂无评论

Managing Patient Admissions in a Neurology Ward

引用

OPERATIONS RESEARCH 2017年第3期65卷 635-656页

作者： Samiedaluie, Saied Kucukyazici, Beste Verter, Vedat Zhang, Dan Univ Alberta Alberta Sch Business Edmonton AB T6G 2R6 Canada McGill Univ Desautels Fac Management Montreal PQ H3A 1G5 Canada Univ Colorado Boulder Leeds Sch Business Boulder CO 80309 USA

We study patient admission policies in a neurology ward where there are multiple types of patients with different medical characteristics. Patients receive specialized care inside the neurology ward and delays in admission to the ward will have negative impact on their health status. The level of this impact varies among patient types and depends on the severity of patients. Patients are also different in terms of arrival rate and length of stay at the ward. The patients normally wait in the emergency department until a ward bed is assigned to them. We formulate this problem as an infinite-horizon average cost dynamic program and propose an efficient approximation scheme to solve large-scale problem instances. The computational results from applying our model to a neurology ward show that dynamic policies generated by our approach can reduce the overall deterioration in patients' health status compared to several alternative policies.

关键词： patient admission neurology ward approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

dynamic Routing: Anticipation of Emission-Sensitive Traffic Management

引用

Transportation Research Procedia 2017年 22卷 419-429页

作者： Felix Köster Marlin W. Ulmer Dirk C. Mattfeld Technische Universität Braunschweig Muhlenpfordtstrasse 23 38106 Braunschweig Germany

Delivery companies are affected by emission-sensitive traffic management systems. These systems are installed in cities to react instantly to emission hot-spots through adapting traffic light programs at intersections. This results in a change of the travel times for the delivery vehicle. In this paper, we model the problem in a dynamic vehicle routing problem with stochastic transition of deterministic travel time matrices. To solve this problem, we apply approximate value iteration, a method of approximate dynamic programming, to anticipate future travel time matrix changes in dynamic routing decisions. We vary the approach in the level of information about the state of the traffic management system. This allows to distinguish the required information for the routing decisions. Further, we compare the dynamic and anticipatory routing policies with a static a priori routing. Computational studies show an improved tour duration of routing with traffic management information over the a priori routing by up to 6.5%. We further show that an efficient representation of the traffic management system status in the approximate value iteration approach is mandatory to achieve sufficient anticipation.

关键词： Vehicle Routing Problem City Logistics Emissions Correlated Travel Times approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Opportunistic Fair Scheduling in Wireless Networks: An approximate dynamic programming Approach

引用

MOBILE NETWORKS & APPLICATIONS 2010年第5期15卷 710-728页

作者： Zhang, Zhi Moola, Sudhir Chong, Edwin K. P. Colorado State Univ Dept Elect & Comp Engn Ft Collins CO 80523 USA

We consider the problem of temporal fair scheduling of queued data transmissions in wireless heterogeneous networks. We deal with both the throughput maximization problem and the delay minimization problem. Taking fairness constraints and the data arrival queues into consideration, we formulate the transmission scheduling problem as a Markov decision process (MDP) with fairness constraints. We study two categories of fairness constraints, namely temporal fairness and utilitarian fairness. We consider two criteria: infinite horizon expected total discounted reward and expected average reward. Applying the dynamic programming approach, we derive and prove explicit optimality equations for the above constrained MDPs, and give corresponding optimal fair scheduling policies based on those equations. A practical stochastic-approximation-type algorithm is applied to calculate the control parameters online in the policies. Furthermore, we develop a novel approximation method-temporal fair rollout-to achieve a tractable computation. Numerical results show that the proposed scheme achieves significant performance improvement for both throughput maximization and delay minimization problems compared with other existing schemes.

关键词： approximate dynamic programming fairness Markov decision process resource allocation scheduling stochastic approximation wireless networks

来源：评论

学校读者我要写书评

暂无评论

Tutorial on Stochastic Optimization in Energy-Part II: An Energy Storage Illustration

引用

IEEE TRANSACTIONS ON POWER SYSTEMS 2016年第2期31卷 1468-1475页

作者： Powell, Warren B. Meisel, Stephan Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08544 USA Univ Munster Dept Informat Syst Munster Germany

In Part I of this tutorial, we provided a canonical modeling framework for sequential, stochastic optimization (control) problems. A major feature of this framework is a clear separation of the process of modeling a problem, versus the design of policies to solve the problem. In Part II, we provide additional discussion behind some of the more subtle concepts such as the construction of a state variable. We illustrate the modeling process using an energy storage problem. We then create five variations of this problem designed to bring out the features of the different policies. The first four of these problems demonstrate that each of the four classes of policies is best for particular problem characteristics. The fifth policy is a hybrid that illustrates the ability to combine the strengths of multiple policy classes.

关键词： approximate dynamic programming dynamic programming energy storage energy systems optimal control reinforcement learning robust optimization stochastic optimization stochastic programming

来源：评论

学校读者我要写书评

暂无评论

Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2016年第2期27卷 444-458页

作者： Wei, Qinglai Song, Ruizhuo Yan, Pengfei Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China

This paper is concerned with a new data-driven zero-sum neuro-optimal control problem for continuous-time unknown nonlinear systems with disturbance. According to the input-output data of the nonlinear system, an effective recurrent neural network is introduced to reconstruct the dynamics of the nonlinear system. Considering the system disturbance as a control input, a two-player zero-sum optimal control problem is established. Adaptive dynamic programming ( ADP) is developed to obtain the optimal control under the worst case of the disturbance. Three single-layer neural networks, including one critic and two action networks, are employed to approximate the performance index function, the optimal control law, and the disturbance, respectively, for facilitating the implementation of the ADP method. Convergence properties of the ADP method are developed to show that the system state will converge to a finite neighborhood of the equilibrium. The weight matrices of the critic and the two action networks are also convergent to finite neighborhoods of their optimal ones. Finally, the simulation results will show the effectiveness of the developed data-driven ADP methods.

关键词： Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming neurodynamic programming nonlinear systems optimal control recurrent neural network (RNN) reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Value Iteration Adaptive dynamic programming for Optimal Control of Discrete-Time Nonlinear Systems

引用

IEEE TRANSACTIONS ON CYBERNETICS 2016年第3期46卷 840-853页

作者： Wei, Qinglai Liu, Derong Lin, Hanquan Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China

In this paper, a value iteration adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon undiscounted optimal control problems for discrete-time nonlinear systems. The present value iteration ADP algorithm permits an arbitrary positive semi-definite function to initialize the algorithm. A novel convergence analysis is developed to guarantee that the iterative value function converges to the optimal performance index function. Initialized by different initial functions, it is proven that the iterative value function will be monotonically nonincreasing, monotonically nondecreasing, or nonmonotonic and will converge to the optimum. In this paper, for the first time, the admissibility properties of the iterative control laws are developed for value iteration algorithms. It is emphasized that new termination criteria are established to guarantee the effectiveness of the iterative control laws. Neural networks are used to approximate the iterative value function and compute the iterative control law, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.

关键词： Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming neural networks neuro-dynamic programming optimal control reinforcement learning value iteration

来源：评论

学校读者我要写书评

暂无评论

Network-Level Infrastructure Management Using approximate dynamic programming

引用

JOURNAL OF INFRASTRUCTURE SYSTEMS 2010年第2期16卷 103-111页

作者： Kuhn, Kenneth D. Univ Canterbury Dept Civil & Nat Resources Engn Christchurch 8041 New Zealand

This research introduces the use of approximate dynamic programming to overcome a variety of limitations of distinct infrastructure management problem formulations. The form, as well as the parameters, of a model specifying the long-term costs associated with alternate infrastructure maintenance policies are learned via simulation. The introduced methodology makes it possible to manage large heterogeneous networks of facilities related by budgetary restrictions and resource constraints as well as by dependencies in maintenance costs or deterioration. In addition, the methodology is particularly well suited to consideration of multiple types of infrastructure condition data at the same time, including continuous-valued data and relevant historical data. Introduced techniques will prove valuable when high-quality deterioration and cost estimation models are available but are ill suited for use in a Markov decision problem framework. Computational studies show that the introduced approach is able to find an optimal solution to a relatively simple infrastructure management problem, and is able to find increasingly good solutions to a more complex problem.

关键词： Infrastructure management approximate dynamic programming Markov decision problem Infrastructure maintenance

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：