检索结果-内蒙古大学图书馆

Convergence analysis of the deep neural networks based globalized dual heuristic programming

AUTOMATICA 2020年 122卷 109222-109222页

作者： Kim, Jong Woo Oh, Tae Hoon Son, Sang Hwan Jeong, Dong Hwi Lee, Jong Min Seoul Natl Univ Engn Dev Res Ctr 1 Gwanak Ro Seoul 08826 South Korea Seoul Natl Univ Inst Chem Proc Sch Chem & Biol Engn 1 Gwanak Ro Seoul 08826 South Korea Texas A&M Univ Artie McFerrin Dept Chem Engn College Stn TX 77845 USA

Globalized dual heuristic programming (GDHP) algorithm is a special form of approximate dynamic programming (ADP) method that solves the Hamilton-Jacobi-Bellman (HJB) equation for the case where the system takes control-affine form subject to the quadratic cost function. This study incorporates the deep neural networks (DNNs) as a function approximator to inherit the advantages of which to express high-dimensional function space. Elementwise error bound of the costate function sequence is newly derived and the convergence property is presented. In the approximated function space, uniformly ultimate boundedness (UUB) condition for the weights of the general multi-layer NNs weights is obtained. It is also proved that under the gradient descent method for solving the moving target regression problem, UUB gradually converges to the value, which exclusively contains the approximation reconstruction error. The proposed method is demonstrated on the continuous reactor control in aims to obtain the control policy for multiple initial states, which justifies the necessity of DNNs structure for such cases. (c) 2020 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Reinforcement learning Deep neural networks Lyapunov stability Nonlinear control

来源：评论

学校读者我要写书评

暂无评论

Data-Driven Control of Unknown Systems: A Linear programming Approach ⁎

引用

IFAC-PapersOnLine 2020年第2期53卷 7-13页

作者： Alexandros Tanzanakis John Lygeros Department of Information Technology and Electrical Engineering ETH Zurich Switzerland

We consider the problem of discounted optimal state-feedback regulation for general unknown deterministic discrete-time systems. It is well known that open-loop instability of systems, non-quadratic cost functions and complex nonlinear dynamics, as well as the on-policy behavior of many reinforcement learning (RL) algorithms, make the design of model-free optimal adaptive controllers a challenging task. We depart from commonly used least-squares and neural network approximation methods in conventional model-free control theory, and propose a novel family of data-driven optimization algorithms based on linear programming, off-policy Q-learning and randomized experience replay. We develop both policy iteration (PI) and value iteration (VI) methods to compute an approximate optimal feedback controller with high precision and without the knowledge of a system model and stage cost function. Simulation studies confirm the effectiveness of the proposed methods.

关键词： linear programming Q-learning approximate dynamic programming data-driven control

来源：评论

学校读者我要写书评

暂无评论

Learning nonlinear robust control as a data-driven zero-sum two-player game for an active suspension system

引用

IFAC-PapersOnLine 2020年第2期53卷 8057-8062页

作者： Mircea-Bogdan Radac Timotei Lala Department of Automation and Applied Informatics Politehnica University of Timisoara Bd. V. Parvan 2 RO-300223 Timisoara Romania

An optimal robust control data-driven learning solution is proposed for an active suspension system. The problem is formulated as a zero-sum two-player differential game (ZS-TP-DG), where the optimal control law and the worst-case disturbance control law must be searched for. The distinctive features of the proposed solution are: a Q-learning-like data-driven model-free (with unknown process dynamics) algorithm relying on collected input-state data from the process; neural networks being used as generic function approximators; validation on an active suspension system that is easily amenable to artificial road profile disturbance generation. The superiority of the ZS-TP-DG controller over another optimal controller learned in a disturbance-free context is validated and proven.

关键词： active suspension system hydraulic actuator neural networks optimal control approximate dynamic programming reinforcement learning two-player zero-sum games

来源：评论

学校读者我要写书评

暂无评论

Online Optimization for Adaptive Scalable Video Transmission in SDN using Kernel-based ADP 13

Online Optimization for Adaptive Scalable Video Transmission...

引用

13th IEEE International Conference on Signal Processing and Communication Systems (ICSPCS)

作者： Chen, Xiang Chen, Shuangwu Ullah, Amin Yu, Chaojie Yang, Jian Univ Sci & Technol China Dept Automat Hefei 230027 Peoples R China

ISBN: (纸本)9781728121949

Video streaming traffic over Internet has rapidly risen to be the main killer of network bandwidth. In order to accommodate the time-varying network conditions, Scalable video coding (SVC) provides a flexible way to transmit a video in different qualities and bit rates. Limited by current best-efforts Internet, video bitrate adaptivity can only conducted on the end side. Recently, the emerging software-defined networking (SDN) technology makes it possible to view the in-network states and control data forwarding. This motivates us to consider the SVC layer selection and multipath routing (video layers go along different path) jointly. The optimization problem is formulated as a Markov Decision Process (MDP) problem. To solve the issue of dimensionality disaster which is common in traditional MDP problem when the state space or action space is large, a kernel based feature extraction is used to transform state-action space to feature space with lower dimensionality. Based on the network features, we construct an approximate value function of long term reward via approximate dynamic programming (ADP) method. Then the optimal policy is derived directly by maximizing the value function. The experiment results show the proposed algorithm achieves a better video quality while keeping a higher convergence speed.

关键词： Scalable video coding Software defined networking Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Multiperiod Stochastic Resource Planning in Professional Services Organizations

引用

DECISION SCIENCES 2019年第6期50卷 1281-1318页

作者： Solomon, Stanislaus Li, Haitao Womer, Keith Santos, Cipriano Southern Illinois Univ Edwardsville Management & Mkt Dept Sch Business Edwardsville IL 62025 USA Univ Missouri St Louis Supply Chain & Analyt Dept Coll Business Adm St Louis MO 63121 USA Gurobi Optimizat Beaverton OR 97008 USA

Resource planning (RP) in a professional service organization matches workforce resources with project tasks while considering a myriad of factors such as skill requirements, service delivery role, skill type, workforce proficiency level, and geographical location. The multiperiod stochastic resource planning studied in this article extends the one-period deterministic resource planning by explicitly coping with both internal resource attrition and project demand uncertainty in a sequential decision-making framework. It allows resource managers to make effective use of their internal resources and identify the need to outsource to external contingent resources. We model the multiperiod stochastic resource planning as a Markov decision process and implement an approximate dynamic programming algorithm to obtain dynamic and adaptive solutions in reasonable computation times. A comprehensive computational study shows that our approximate dynamic programming algorithm achieves higher profitability and internal resource utilization compared to the rolling horizon approach used as a benchmark.

关键词： approximate dynamic programming Markov Decision Process Project Management Resource Planning Uncertainty Workforce Optimization

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming solutions with a single network adaptive critic for a class of nonlinear systems

引用

控制理论与应用（英文版） 2011年第3期9卷 370-380页

作者： S.N.BALAKRISHNAN Department of Mechanical and Aerospace Engineering Missouri University of Science and Technology

approximate dynamic programming(ADP) formulation implemented with an adaptive critic(AC)-based neural network(NN) structure has evolved as a powerful technique for solving the Hamilton-Jacobi-Bellman(HJB) *** interest in ADP and the AC solutions are escalating with time,there is a dire need to consider possible enabling factors for their implementations.A typical AC structure consists of two interacting NNs,which is computationally *** this paper,a new architecture,called the ’cost-function-based single network adaptive critic(J-SNAC)’ is presented,which eliminates one of the networks in a typical AC *** approach is applicable to a wide class of nonlinear systems in *** order to demonstrate the benefits and the control synthesis with the J-SNAC,two problems have been solved with the AC and the J-SNAC *** are presented,which show savings of about 50% of the computational costs by J-SNAC while having the same accuracy levels of the dual network structure in solving for optimal ***,convergence of the J-SNAC iterations,which reduces to a least-squares problem,is discussed;for linear systems,the iterative process is shown to reduce to solving the familiar algebraic Ricatti equation.

关键词： approximate dynamic programming Optimal control Nonlinear control Adaptive critic Cost-functionbased single network adaptive critic J-SNAC architecture

来源：评论

学校读者我要写书评

暂无评论

Multiobjective Home Appliances Scheduling Considering Customer Thermal Discomfort: A Multistep Look-ahead ADP-Based Approach

Multiobjective Home Appliances Scheduling Considering Custom...

引用

IEEE Milan PowerTech Conference

作者： Jeddi, Babak Mishra, Yateendra Ledwich, Gerard Queensland Univ Technol QUT Sch Elect Engn & Comp Sci Brisbane Qld Australia

ISBN: (纸本)9781538647226

This paper proposes a multiobjective home energy management unit (MO-HEMU) to balance the electricity payment and thermal discomfort of a household by properly scheduling devices in a time-varying price environment. The thermal discomfort is measured by the deviation of indoor and hot water temperature from the user's ideal temperature. The home devices include solar Photovoltaics, a battery storage system, deferrable, and thermal appliances. The proposed MO-HEMU is formulated as a dynamic program and a method based on the approximated dynamic programming is used as the scheduling algorithm. The developed method, called multistep look-ahead algorithm, is an iterative algorithm that overcomes the curse of dimensionality of the exact DP by choosing a decision based on a limited number of stages ahead. The effectiveness of the proposed model is investigated through numerical simulations. The proposed MO-HEMU enables customers to find the desired trade-off between electricity payment and a discomfort level.

关键词： Appliances scheduling approximate dynamic programming home energy management multiobjective optimization thermal discomfort

来源：评论

学校读者我要写书评

暂无评论

A Reinforcement Learning Method for Traffic Signal Control at an Isolated Intersection with Pedestrian Flows 19

A Reinforcement Learning Method for Traffic Signal Control a...

引用

19th COTA International Conference of Transportation Professionals (CICTP) - Transportation in China 2025

作者： Yin, Biao Menendez, Monica New York Univ Abu Dhabi Div Engn POB 129188 Abu Dhabi U Arab Emirates NYU Tandon Sch Engn New York NY USA

ISBN: (数字)9780784482292

ISBN: (纸本)9780784482292

In the paper, we propose a model-based reinforcement learning algorithm, i.e., approximate dynamic programming for signal control at isolated intersections with mixed traffic: vehicles and pedestrians. The integrated optimization problem is formulated by the discrete-time dynamic process. The system state is represented by the combination of weighted vehicle queue lengths and the number of waiting pedestrians. The system action is generated in each decision step by the proposed algorithm. To solve the computation issue in conventional dynamic programming, the proposed algorithm adopts a linear approximation function that helps to quickly obtain a near-optimal solution. In simulation, we extract traffic information from the traffic simulator SUMO. The on-line traffic information is provided for the algorithm to make a signal decision. After testing various scenarios, results show that the proposed algorithm has potential control performance. We also reveal the delay changes with different weights assigned to the vehicle and pedestrian components.

关键词： approximate dynamic programming Signal control Intersection Pedestrians

来源：评论

学校读者我要写书评

暂无评论

A proactive transfer policy for critical patient flow management

引用

HEALTH CARE MANAGEMENT SCIENCE 2019年第2期22卷 287-303页

作者： Gonzalez, Jaime Ferrer, Juan-Carlos Cataldo, Alejandro Rojas, Luis Pontificia Univ Catolica Chile Sch Engn Vicuna Mackenna 4860 Santiago Chile Pontificia Univ Catolica Chile Sch Med Libertador Bernardo OHiggins 340 Santiago Chile

Hospital emergency departments are often overcrowded, resulting in long wait times and a public perception of poor attention. Delays in transferring patients needing further treatment increases emergency department congestion, has negative impacts on their health and may increase their mortality rates. A model built around a Markov decision process is proposed to improve the efficiency of patient flows between the emergency department and other hospital units. With each day divided into time periods, the formulation estimates bed demand for the next period as the basis for determining a proactive rather than reactive transfer decision policy. Due to the high dimensionality of the optimization problem involved, an approximate dynamic programming approach is used to derive an approximation of the optimal decision policy, which indicates that a certain number of beds should be kept free in the different units as a function of the next period demand estimate. Testing the model on two instances of different sizes demonstrates that the optimal number of patient transfers between units changes when the emergency patient arrival rate for transfer to other units changes at a single unit, but remains stable if the change is proportionally the same for all units. In a simulation using real data for a hospital in Chile, significant improvements are achieved by the model in key emergency department performance indicators such as patient wait times (reduction higher than 50%), patient capacity (21% increase) and queue abandonment (from 7% down to less than 1%).

关键词： Markov decision process approximate dynamic programming Emergency department Critical care beds Patient flow

来源：评论

学校读者我要写书评

暂无评论

Strategic Location and Dispatch Management of Assets in a Military Medical Evacuation Enterprise

Strategic Location and Dispatch Management of Assets in a Mi...

引用

作者： Phillip R. Jenkins Air University

学位级别：博士

This dissertation considers the importance of optimizing deployed military med- ical evacuation (MEDEVAC) systems and utilizes operations research techniques to develop models that allow military medical planners to analyze different strategies regarding the management of MEDEVAC assets in a deployed environment. For op- timization models relating to selected subproblems of the MEDEVAC enterprise, the work herein leverages integer programming, multi-objective optimization, Markov de- cision processes (MDPs), approximate dynamic programming (ADP), and machine learning, as appropriate, to identify relevant insights for aerial MEDEVAC operations. This research is conducted in the form of three related research components: the first component develops models for optimal MEDEVAC location within an enterprise of assets, whereas the second and third components seek optimal MEDEVAC dis- patching policies for a set of established assets via a sequence of models of increasing operational complexity and corresponding solution methods. Determining where to locate mobile aeromedical staging facilities (MASFs) as well as identifying how many aeromedical helicopters to allocate to each MASF, commonly referred to as the MEDEVAC location-allocation problem, is vital to the success of a deployed MEDEVAC system. An integer mathematical programming formulation is developed to determine the location and allocation of MEDEVAC as- sets over the phases of a military deployment to support operations ranging from peacekeeping through combat to post-combat resolution. The model seeks to address the multi-objective problem of maximizing the total expected coverage of demand as a measure of solution effectiveness, minimizing the maximum number of located MASFs in any deployment phase as a measure of solution efficiency, and minimizing the total number of MASF relocations throughout the deployment as a measure of solution robustness. Moreover, the model utilizes the ε-constraint Method to assess

关键词： Military medical evacuation (MEDEVAC) integer programming multi-objective optimization Markov decision process approximate dynamic programming machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：