检索结果-内蒙古大学图书馆

Synchronous optimal control method for nonlinear systems with saturating actuators and unknown dynamics using off-policy integral reinforcement learning

引用

NEUROCOMPUTING 2019年 356卷 162-169页

作者： Zhang, Zenglian Song, Ruizhuo Cao, Min Univ Sci & Technol Beijing Donlinks Sch Econ & Management Beijing 100083 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China China Energy Engn Investment Corp Ltd Beijing 100022 Peoples R China

The present study establishes an approximate optimal critic learning algorithm, based on the single-network integral reinforcement learning (IRL) algorithm and intends to solve the optimal control problem for an unknown nonlinear system with saturating actuators. The value function is formulated through building generalized nonquadratic functions. In order to solve the Hamilton-Jacobi-Bellman (HJB) equation, a novel optimal scheme for the control approximation, based on the off-policy iteration is presented. Moreover, the single-neural network implementation procedure is introduced to complete the iteration algorithm. The synchronous IRL policy iteration is proposed to update the weight of the critic neural network. Finally, reasonable simulation results are provided for confirming the effectiveness of the proposed optimal approximation control technique in solving equations for a linear and oscillating systems. (C) 2019 Elsevier B.V. All rights reserved.

关键词： approximate dynamic programming Adaptive dynamic programming Off-policy Integral reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Neural-network-based learning algorithms for cooperative games of discrete-time multi-player systems with control constraints via adaptive dynamic programming

引用

NEUROCOMPUTING 2019年 344卷 13-19页

作者： Jiang, He Zhang, Huaguang Xie, Xiangpeng Han, Ji Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China Nanjing Univ Posts & Telecommun Inst Adv Technol Nanjing 210003 Peoples R China

Adaptive dynamic programming (ADP), an important branch of reinforcement learning, is a powerful tool in solving various optimal control problems. However, the cooperative game issues of discrete-time multiplayer systems with control constraints have rarely been investigated in this field. In order to address this issue, a novel policy iteration (PI) algorithm is proposed based on ADP technique, and its associated convergence analysis is also studied in this brief paper. For the proposed PI algorithm, an online neural network (NN) implementation scheme with multiple-network structure is presented. In the online NN-based learning algorithm, critic network, constrained actor networks and unconstrained actor networks are employed to approximate the value function, constrained and unconstrained control policies, respectively, and the NN weight updating laws are designed based on the gradient descent method. Finally, a numerical simulation example is illustrated to show the effectiveness. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Adaptive dynamic programming approximate dynamic programming Reinforcement learning Neural networks

来源：评论

学校读者我要写书评

暂无评论

Two-level task scheduling with multi-objectives in geo-distributed and large-scale SaaS cloud

引用

WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS 2019年第6期22卷 2291-2319页

作者： Zhang, Puheng Ma, Xiao Xiao, Yanping Li, Wenzhuo Lin, Chuang Logist Res Inst Sci & Technol Beijing Peoples R China Tsinghua Univ Dept Comp Sci & Technol Tsinghua Natl Lab Informat Sci & Technol Beijing Peoples R China

With the exploding of data-intensive web applications and requests (tasks), geo-distributed and large-scale data centers (DCs) are widely deployed in Software as a Service (SaaS) cloud, but server failures continue to grow at the same time. In this context, task scheduling problems become more intricate and both scheduling quality and scheduling speed raise further concerns. In this paper, we first propose a virtualized & monitoring SaaS model with predictive maintenance to minimize the costs of fault tolerance. Then with the monitored and predicted available states of servers, we focus on dynamic real-time task scheduling in geo-distributed and large-scale DCs with heterogeneous servers. Multiple objectives, including the long-term performance benefits, energy and communication costs, are taken into consideration in order to improve scheduling quality. For inter-DC and intra-DC task scheduling, two dynamic programming problems are formulated respectively, but there exists the problem that both state and action spaces are too large to be solved by simple iterations. To address this issue, we introduce the idea of reinforcement learning theory into solving traditional stochastic dynamic programming problems in the large-scale SaaS cloud, and put forward a cascaded two-level (inter-DC and intra-DC level) approximate dynamic programming (ADP) task-scheduling algorithm. The computation complexity can be significantly reduced and scheduling speed can be greatly improved. Finally, we conduct experiments with both random simulation data and Google cloud trace-logs. QoS evaluations and comparisons demonstrate that two ADP algorithms can work cooperatively, and our two-level ADP algorithm is more effective under large quantity of bursty requests.

关键词： Multi-objective optimization SaaS cloud Data center Task scheduling approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

dynamic Lookahead Policies for Stochastic-dynamic Inventory Routing in Bike Sharing Systems

引用

COMPUTERS & OPERATIONS RESEARCH 2019年 106卷 260-279页

作者： Brinkmann, Jan Ulmer, Marlin W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig D-38106 Braunschweig Germany

We present the stochastic-dynamic inventory routing problem for bike sharing systems (SDIRP). The objective of the SDIRP is to avoid unsatisfied demand by dynamically relocating bikes during the day. To anticipate potential future demands in the current inventory decisions, we present a dynamic lookahead policy (DLA). The policy simulates future demand over a predefined horizon. Because the heterogeneous demand patterns over the course of the day, the DLA horizons are time-dependent and autonomously parametrized by means of value function approximation, a method of approximate dynamic programming. We compare the DLA with conventional relocation strategies from the literature and lookahead policies with static horizons. Our study based on real-world data by the bike sharing system of Min-neapolis (Minnesota, USA) reveals the benefits of both anticipation by lookaheads as well as the time-dependent horizons of the DLA. We additionally show how the DLA is able to autonomously adapt to the demand patterns. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Bike sharing approximate dynamic programming dynamic vehicle routing Lookahead policies Value function approximation Policy search

来源：评论

学校读者我要写书评

暂无评论

New heuristics for the RCPSP with multiple overlapping modes

引用

COMPUTERS & INDUSTRIAL ENGINEERING 2019年 131卷 146-156页

作者： Chu, Zihao Xu, Zhe Li, Haitao Beihang Univ Sch Econ & Management Beijing 100191 Peoples R China Univ Missouri Coll Business Adm Supply Chain & Analyt Dept St Louis MO 63121 USA

Activity overlapping in project management and concurrent engineering allows a downstream activity to start before the end of its upstream activity based on some preliminary information and feedback. Because the preliminary information can be issued at multiple stages during an upstream activity's execution, the downstream activity may have multiple overlapping modes. Project scheduling with activity overlapping has a wide range of applications in manufacturing, R&D, construction and professional service industries. An optimal overlapping plan may help project manager accelerate project progress effectively with improved rate of investment. In this research, we study a resource-constrained project scheduling problem with multiple overlapping modes, which is NP-hard. To obtain high-quality near-optimal solutions, we formulate it as a dynamic program (DP), and develop a rollout policy based approximate dynamic programming (ADP) algorithm to obtain near-optimal policy efficiently. Our rollout policy sequentially improves over a priority-rule base policy. A rollout procedure based on pre-processing of overlapping options is also devised which we call pre-rollout. The pre-rollout improves over the rollout in some instances with moderately more computational time. A comprehensive experiment is performed on the PSPLIB benchmark instances. Computational results show that our rollout policy and the pre-rollout procedure significantly outperform the exact integer linear programming (ILP) solutions in both solution quality and computational time for medium and large instances.

关键词： Resource-constrained project scheduling Concurrent engineering Multiple overlapping modes approximate dynamic programming Rollout policy

来源：评论

学校读者我要写书评

暂无评论

Integrated scheduling of drayage and long-haul operations in synchromodal transport

引用

FLEXIBLE SERVICES AND MANUFACTURING JOURNAL 2019年第3期31卷 763-806页

作者： Rivera, Arturo E. Perez Mes, Martijn R. K. Univ Twente Dept Ind Engn & Business Informat Syst POB 217 NL-7500 AE Enschede Netherlands

We study the problem of the integrated scheduling of drayage operations and long-haul transport in synchromodality. Although different in time span and characteristics of execution, these two processes have an impact on each other and their interaction has a direct influence on the overall performance of the transport network over time. We propose a simulation based integration of a Mixed-Integer Linear programming model for the drayage operations and a Markov Decision Process model for the long-haul transport. We analyze the interfaces between these models, outline the challenges of integrating them, and design a heuristic approach to the simulation based integration. In a series of numerical experiments, we evaluate the cost savings compared to a non-integrated approach, using various transport network configurations. We show that our approach achieves average cost savings between 4 and 24% on networks with a majority of pre-haulage freights. Furthermore, we discuss limitations of our model and experiments, and provide guidelines for further research for the integrated scheduling of drayage and long-haul operations in synchromodal transport.

关键词： Synchromodal transport Drayage Long-haul Matheuristic approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

The Delivery Dispatching Problem with Time Windows for Urban Consolidation Centers

引用

TRANSPORTATION SCIENCE 2019年第1期53卷 203-221页

作者： van Heeswijk, W. J. A. Mes, M. R. K. Schutten, J. M. J. Univ Twente Dept Ind Engn & Business Informat Syst NL-7522 NB Enschede Netherlands

This paper addresses the dispatching problem faced by an urban consolidation center. The center receives orders according to a stochastic arrival process and dispatches them in batches for the last-mile distribution. The operator of the center aims to find the cost-minimizing consolidation policy, depending on the orders at hand, preannounced orders, and stochastic arrivals. We present this problem as a variant of the delivery dispatching problem that includes dispatch windows and define a corresponding Markov decision model. Larger instances of the problem suffer from intractably large state-, outcome-, and action spaces. We propose an approximate dynamic programming (ADP) algorithm that can handle such instances, using a linear value function approximation to estimate the downstream costs. To design the value function approximation, we construct various sets of basis functions, numerically evaluate their suitability, and discuss the properties of good basis functions for the dispatching problem. Numerical experiments on toy-sized instances show that the best set of basis functions approximates the optimal values with an error of less than 3%. To cope with large action spaces, we formulate an integer linear program to be used within our ADP algorithm. We evaluate the performance of ADP policies against four benchmark policies: two heuristic policies, a direct cost minimization policy, and a post-decision rollout policy. We test the performance of ADP on a variety of networks. ADP consistently outperforms the benchmark policies, performing particularly well when there is sufficient flexibility in dispatch times.

关键词： urban distribution transport planning freight consolidation approximate dynamic programming dispatching problem

来源：评论

学校读者我要写书评

暂无评论

Energy Management of PV-Storage Systems: Policy Approximations Using Machine Learning

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2019年第1期15卷 257-265页

作者： Keerthisinghe, Chanaka Chapman, Archie C. Verbic, Gregor Univ Sydney Sch Elect & Informat Engn Sydney NSW 2006 Australia Univ Washington Dept Elect Engn Seattle WA 98195 USA

In this paper, we propose a policy function approximation (PFA) algorithm using machine learning to effectively control photovoltaic (PV)-storage systems. The algorithm uses an offline policy planning stage and an online policy execution stage. In the planning stage, a suitable machine learning technique is used to generate models that map states (inputs) and decisions (outputs) using training data. The training dataset is generated by solving a deterministic smart home energy management problem using a suitable optimization technique [e.g., mathematical programming or dynamic programming (DP)]. In the execution stage, the model generated by the machine learning algorithm is then used to generate fast real-time decisions. Since the decisions can be made in real-time, the policy can rely on up-to-date information on PV output, electrical demand, and battery state of charge. Moreover, we can use PFA models over a long period of time (i.e., months) without having to update them but still obtain similar quality solutions. Our results show that the solutions from the PFAs are close to the best solutions obtained using DP and approximate DP, which have the drawback of requiring an optimization problem to be solved before the beginning of each day or as new information on demand or PV becomes available.

关键词： approximate dynamic programming demand response distributed energy resources dynamic programming machine learning policy function approximations smart home energy management

来源：评论

学校读者我要写书评

暂无评论

Four actor-critic structures and algorithms for nonlinear multi-input multi-output system

引用

NEUROCOMPUTING 2019年 330卷 172-187页

作者： Huang, Zhijian Zhang, Yanyan Liu, Yihua Zhang, Guichen Shanghai Maritime Univ Lab Intelligent Control & Computat Shanghai 201306 Peoples R China Tongji Univ Peoples Hosp 10 Shanghai 200072 Peoples R China Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA

The action-dependent heuristic approximate dynamic (ADHDP) for nonlinear multi-input multi-output (MIMO) system needs different forms to adapt to variable practical objects. Due to some inappropriate network structure or training algorithm, unsuccessful designs or undesirable control effect is common in reality. Thus, at first, this paper addresses the chain rule problem of the compound derivative in training the nonlinear MIMO ADHDP. Then, this paper researches and proposes four actor-critic algorithms systematically according to four typical nonlinear systems. That is, the action-network extension, the sub-network, the cascaded action-network and the combined method. To illustrate the four methods, their detailed structures, derivation procedures and training algorithms are derived. The Lyapunov stability for the nonlinear MIMO ADHDP is proved as well. Through examples of an idling engine and aircraft controlling, the simulation results show the effectiveness of these methods. Besides, the property, advantages, disadvantages and the applicability of these methods are compared and highlighted. The four methods can be used to meet the design requirement of almost all the nonlinear MIMO ADHDP control systems. For incoming scholars in search of a nonlinear MIMO ADHDP to achieve the best control effect, the four actor-critic structures and algorithms can be a reference. (C) 2018 Elsevier B.V. All rights reserved.

关键词： approximate dynamic programming Action-dependent heuristic approximate dynamic Nonlinear Multi-input-multi-output Structure Training Neural network

来源：评论

学校读者我要写书评

暂无评论

Neural-Network-Based Robust Control Schemes for Nonlinear Multiplayer Systems With Uncertainties via Adaptive dynamic programming

引用

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2019年第3期49卷 579-588页

作者： Jiang, He Zhang, Huaguang Luo, Yanhong Han, Ji Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China

This paper investigates the robust control issues of nonlinear multiplayer systems by utilizing adaptive dynamic programming (ADP) methods and fills a gap in the ADP field, where actuator uncertainties for multiplayer systems are still not addressed. Two types of actuator uncertainties including bounded nonlinear perturbation and unknown constant actuator fault are taken into consideration. First, a data-driven reinforcement learning (RL) approach is derived to learn the optimal solutions of multiplayer nonzero-sum games. Then, based on the obtained optimal control policies, two robust control schemes are developed to handle these two different types of uncertainties, respectively, and the associated stability analysis is also provided. To implement the proposed iterative RL approach, a single neural network (NN) architecture with least-square-based updating law is given, which reduces the computation burden compared with the traditional dual NN architecture. Finally, two numerical examples are shown to test the feasibility of our proposed schemes.

关键词： Adaptive dynamic programming (ADP) approximate dynamic programming neural network (NN) reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：