检索结果-内蒙古大学图书馆

An approximate dynamic programming framework for modeling global climate policy under decision-dependent uncertainty

引用

COMPUTATIONAL MANAGEMENT SCIENCE 2012年第3期9卷 339-362页

作者： Webster, Mort Santen, Nidhi Parpas, Panos MIT Engn Syst Div 77 Massachusetts Ave Cambridge MA 02139 USA Imperial Coll London Dept Comp London SW7 2AZ England

Analyses of global climate policy as a sequential decision under uncertainty have been severely restricted by dimensionality and computational burdens. Therefore, they have limited the number of decision stages, discrete actions, or number and type of uncertainties considered. In particular, two common simplifications are the use of two-stage models to approximate a multi-stage problem and exogenous formulations for inherently endogenous or decision-dependent uncertainties (in which the shock at time t+1 depends on the decision made at time t). In this paper, we present a stochastic dynamic programming formulation of the dynamic Integrated Model of Climate and the Economy (DICE), and the application of approximate dynamic programming techniques to numerically solve for the optimal policy under uncertain and decision-dependent technological change in a multi-stage setting. We compare numerical results using two alternative value function approximation approaches, one parametric and one non-parametric. We showt hat increasing the variance of a symmetric mean-preserving uncertainty in abatement costs leads to higher optimal first-stage emission controls, but the effect is negligible when the uncertainty is exogenous. In contrast, the impact of decision-dependent cost uncertainty, a crude approximation of technology R&D, on optimal control is much larger, leading to higher control rates (lower emissions). Further, we demonstrate that themagnitude of this effect grows with the number of decision stages represented, suggesting that for decision-dependent phenomena, the conventional two-stage approximation will lead to an underestimate of the effect of uncertainty.

关键词： Climate policy analysis approximate dynamic programming Decision dependent uncertainty Stochastic dynamic programming Endogenous uncertainty

来源：评论

学校读者我要写书评

暂无评论

Adaptive network traffic control with an integrated model-based and data-driven approach and a decentralised solution method*

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2021年 128卷 103154-103154页

作者： Su, Z. C. Chow, Andy H. F. Zhong, R. X. Univ Hong Kong Dept Architecture & Civil Engn Hong Kong Peoples R China Sun Yat Sen Univ Sch Intelligent Syst Engn Guangzhou Peoples R China

This paper presents an adaptive traffic controller for stochastic road networks with an integrated model-based and data-driven solution framework. The model-based optimisation component operates based upon an underlying kinematic wave model driven by stochastic demand within a prediction horizon. The data-driven optimisation component operates based upon an approximate dynamic programming (ADP) formulation which approximates the state-control interactions over future stages with a parametric approximator. The approximator reduces the computational complexity of the adaptive control problem by parameterising the state and decision space. The parametric approximator is to be iteratively updated with online feeding of realisations of traffic states via a temporal difference (TD) learning process. Our results reveal that incorporation of the model-based component facilitates the training of the ADP-based state approximator, and hence improve the overall performance of the control system. We further develop a decentralised solution approach in which individual intersections are allowed to derive their own control policies in an asynchronous manner. The data-driven ADP approximator would serve as a central agent coordinating the control policies derived at individual intersections in the network. This is shown to be able to improve and stabilise the performance of the overall control system even under congested conditions. This is a significant progress in adaptive control system design with use of decentralised optimisation techniques. The present study contributes to the adaptive network traffic control with uncertainties through use of advanced modelling and optimisation methods.

关键词： Adaptive network traffic control Markov decision process approximate dynamic programming Acyclic signal timing Decentralised optimisation

来源：评论

学校读者我要写书评

暂无评论

dynamic Programs with Shared Resources and Signals: dynamic Fluid Policies and Asymptotic Optimality

引用

OPERATIONS RESEARCH 2021年第5期70卷 3015页

作者： Brown, David B. Zhang, Jingwei Duke Univ Fuqua Sch Business Durham NC 27708 USA

We consider a sequential decision problem involving shared resources and signals in which a decision maker repeatedly observes some exogenous information (the signal), modeled as a finite-state Markov process, then allocates a limited amount of a shared resource across a set of projects. The framework includes a number of applications and generalizes Markovian multiarmed bandit problems by (a) incorporating exogenous information through the signal and (b) allowing for more general resource allocation decisions. Such problems are naturally formulated as stochastic dynamic programs (DPs), but solving the DP is impractical unless the number of projects is small. In this paper, we develop a Lagrangian relaxation and a DP formulation of the corresponding fluid relaxation-a dynamic fluid relaxation-that provide upper bounds on the optimal value function as well as a feasible policy. We develop an iterative primal-dual algorithm for solving the dynamic fluid relaxation and analyze the performance of the feasible dynamic fluid policy. Our performance analysis implies, under mild conditions, that the dynamic fluid relaxation bound and feasible policy are asymptotically optimal as the number of projects grows large. Our Lagrangian relaxation uses Lagrange multipliers that depend on the history of past signals in each period: we show that the bounds and analogous policies using restricted forms of Lagrange multipliers (e.g., only depending on the current signal state in each period) in general lead to a performance gap that is linear in the number of projects and thus are not asymptotically optimal in the regime of many projects. We demonstrate the model and results in two applications: (i) a dynamic capital budgeting problem and (ii) a multilocation inventory management problem with limited production capacity and demands that are correlated across locations by a changing market state.

关键词： weakly coupled stochastic dynamic programs Lagrangian relaxations approximate dynamic programming capital budgeting inventory control

来源：评论

学校读者我要写书评

暂无评论

Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2021年 127卷 106657-106657页

作者： Saadatmand, Sepehr Shamsi, Pourya Ferdowsi, Mehdi Missouri Univ Sci & Technol Dept Elect & Comp Engn Rolla MO 65409 USA

In this paper, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

关键词： Adaptive critic design approximate dynamic programming DC/AC converters Heuristic dynamic programming Machine learning Reinforcement learning Virtual inertia Synchronverter

来源：评论

学校读者我要写书评

暂无评论

Decentralized Formation Shape Control of UAV Swarm using dynamic programming 29

Decentralized Formation Shape Control of UAV Swarm using Dyn...

引用

Conference on Signal Processing, Sensor/Information Fusion, and Target Recognition XXIX

作者： Azam, Md Ali Ragi, Shankarachary South Dakota Sch Mines & Technol Dept Elect Engn Rapid City SD 57701 USA

ISBN: (纸本)9781510636248;9781510636231

Formation control of unmanned aerial vehicles (UAVs) has many applications including target tracking, surveillance, terrain mapping, precision agriculture, etc. Although many centralized control methods (single command center/computer controlling the UAVs) exist, there are no standard decentralized control frameworks in the literature. In this paper, we present a novel UAV swarm formation control approach based on a decision theoretic approach. Specifically, we pose the decentralized swarm motion control problem as a Decentralized Markov Decision Process (Dec-MDP). Here, the objective is to drive the swarm from an initial geographical region to another geographical region where the swarm must lie on a certain geometrical surface (e.g., surface of a sphere). As most decision theoretic formulations suffer from the curse of dimensionality, we adapt an approximate dynamic programming method called nominal belief-state optimization (NBO) to solve the formation control problem approximately. We perform simulation studies in MATLAB to validate the performance of the algorithms.

关键词： Swarm intelligence formation control decentralized Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Data-based and secure switched cyber-physical systems

引用

SYSTEMS & CONTROL LETTERS 2021年 148卷 104826-104826页

作者： Zhai, Lijing Vamvoudakis, Kyriakos G. Georgia Inst Technol Daniel Guggenheim Sch Aerosp Engn Atlanta GA 30332 USA

In this paper, we develop a completely model-free moving target defense framework for the detection and mitigation of sensor and/or actuator attacks in cyber-physical systems with dynamics that evolve in discrete-time. We incorporate an intrusion detection mechanism based on an approximate dynamic programming technique that learns the policies for optimal regulation and optimal tracking while simultaneously defending against actuator and sensor attacks in a model-free fashion. Switching rules are leveraged to force proactive and reactive defense mechanisms as well as, guarantee the stability of the equilibrium point. Finally, as a case study, we apply the proposed moving target defense framework to a DC-DC converter that is used in electric vehicles. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Cyber-physical systems approximate dynamic programming Moving target defense Learning Security Switched systems

来源：评论

学校读者我要写书评

暂无评论

dynamic Resource Management Algorithms for Complex Systems and Novel Approaches to Adaptive Kalman Filtering

Dynamic Resource Management Algorithms for Complex Systems a...

引用

作者： Zhang, Lingyi University of Connecticut

学位级别：Ph.D., Doctor of Philosophy

This thesis considers three combinatorial optimization problems of substantial practical importance. First, a new approach to efficiently obtain a large number of ranked solutions to a 3-dimensional assignment problem is presented, and is applied to generate fuel assembly loading patterns. Second, we formulate the problem of dynamically scheduling maritime surveillance assets, and solve it using branch-and-cut and approximate dynamic programming (ADP) with rollout, and investigate the tradeoffs between the two. Third, a multi-objective ship routing problem is also investigated, where we propose a solution combining approximate dynamic programming techniques and clustering techniques to contain the computational and storage complexity. Lastly, this dissertation develops a seminal approach to adaptive Kalman filtering via the use of post-fit residuals given data samples -- an approach not yet discussed prior to this work.

关键词： Kalman filter optimization algorithms covariance estimation noise covariance approximate dynamic programming rollout branch-and-cut

来源：评论

学校读者我要写书评

暂无评论

Understanding Consumer dynamic Decision Making Under Competing Loyalty Programs

引用

JOURNAL OF MARKETING RESEARCH 2020年第3期57卷 422-444页

作者： Liu, Jia Ansari, Asim Hong Kong Univ Sci & Technol Mkt Hong Kong Peoples R China Columbia Univ Columbia Business Sch Mkt Div Mkt New York NY 10027 USA

The authors develop an incentive-aligned experimental paradigm to study how consumer purchase dynamics are affected by the interplay between competing firms' loyalty programs and their pricing and promotional strategies. In this experiment, participants made sequential choices between two competing airlines in a stylized frequent traveler task for which an optimal dynamic decision policy can be numerically computed. The authors find that, on average, participants are able to partially realize the long-term benefits from loyalty programs, though most are sensitive to price. They also find that participants' preferences and levels of bounded rationality depend on the nature of the competitive environment, the particular state of each decision scenario, and the type of optimal action. Accordingly, the authors use an approximate dynamic programming model to incorporate boundedly rational decision making. The model classifies participants into five segments that exhibit variation in their performance and decision strategies. Importantly, they find that participants are able to adapt their decision strategies to the environment they face, and thus the overall market outcome and the performance of each firm are influenced by both the competitive environment and the assumption on the extent of consumer optimality.

关键词： approximate dynamic programming bounded rationality competition experiments loyalty programs promotions

来源：评论

学校读者我要写书评

暂无评论

Reinforcement-Learning-based Optimization for Day-ahead Flexibility Extraction in Battery Pools 21st

Reinforcement-Learning-based Optimization for Day-ahead Flex...

引用

21st IFAC World Congress on Automatic Control - Meeting Societal Challenges

作者： Chasparis, Georgios C. Lettner, Christian Software Competence Ctr Hagenberg GmbH Softwarepk 21 A-4232 Hagenberg Austria

We address the problem of trading energy flexibility, derived from pools of residential Photovoltaic and battery-storage systems, to the Day-ahead electricity market. By flexibility, we imply any additional energy that can be stored to or withdrawn from the participating batteries/households at a given time during the next day. The optimization variables include the selection/activation of a subset of participating batteries and the amount of flexibility that should be extracted. Furthermore, the optimization objective corresponds to the expected forecast revenues that can be generated by trading this flexibility to the Day-ahead electricity market. Given the high computationally complexity of a full scale optimization in the case of a large number of participating batteries, we propose a reinforcement-learning-based methodology, which admits linear complexity with the number of participating batteries. The proposed methodology advances prior work with respect to the integration of a large number of batteries. Furthermore, it extends prior work of the authors with respect to providing analytical performance guarantees in comparison with the baseline/nominal operation of the battery. Finally, we compare through simulations the performance of the proposed method with a Linear-programming-based optimization that provides the exact optimum. Copyright (C) 2020 The Authors.

关键词： Control of renewable energy resources Smart grids approximate dynamic programming Day-ahead spot electricity market

来源：评论

学校读者我要写书评

暂无评论

Improving System-of-Systems Agility through dynamic Reconfiguration

Improving System-of-Systems Agility through Dynamic Reconfig...

引用

IEEE International Conference on Systems, Man, and Cybernetics (SMC)

作者： Fang, Zhemei Liao, Jingjing Zhou, Xiaozhou Huazhong Univ Sci & Technol Sch Artificial Intelligence & Automat Natl Key Lab Sci & Technol Multispectral Informat Wuhan 430074 Peoples R China Wuhan Maritime Commun Res Inst Wuhan 430205 Peoples R China

ISBN: (纸本)9781728185262

Most System-of-Systems (SoS) problems face a highly dynamic, volatile, and uncertain environment. How to cope with the unpredictable and volatile changes is one of the key challenges for SoS practitioners. Agility, as an informative feature indicating an SoS's capability of effecting, coping with and exploiting changes, has not received sufficient attention in the context of SoS. This paper proposes a dynamic reconfiguration method that aims to improve the SoS agility from the perspectives of responding quickly to failures and recovering some amount of the SoS capability. This paper employs approximate dynamic programming method to compute the dynamic reconfiguration decisions that allow failed or degraded systems to be removed and the function-capability allocation to be changed quickly. A naval warfare SoS example takes a preliminary step towards demonstrating the effectiveness of the dynamic reconfiguration method, along with flexible architecture, in achieving the SoS agility in terms of responsiveness and resilience.

关键词： system-of-systems agility dynamic reconfiguration hierarchical representation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：