检索结果-内蒙古大学图书馆

Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2021年 127卷

作者： Saadatmand, Sepehr Shamsi, Pourya Ferdowsi, Mehdi Missouri Univ Sci & Technol Dept Elect & Comp Engn Rolla MO 65409 USA

In this paper, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

关键词： Adaptive critic design approximate dynamic programming DC/AC converters Heuristic dynamic programming Machine learning Reinforcement learning Virtual inertia Synchronverter

来源：评论

学校读者我要写书评

暂无评论

Reinforcement-Learning-based Optimization for Day-ahead Flexibility Extraction in Battery Pools 21st

Reinforcement-Learning-based Optimization for Day-ahead Flex...

引用

21st IFAC World Congress on Automatic Control - Meeting Societal Challenges

作者： Chasparis, Georgios C. Lettner, Christian Software Competence Ctr Hagenberg GmbH Softwarepk 21 A-4232 Hagenberg Austria

We address the problem of trading energy flexibility, derived from pools of residential Photovoltaic and battery-storage systems, to the Day-ahead electricity market. By flexibility, we imply any additional energy that can be stored to or withdrawn from the participating batteries/households at a given time during the next day. The optimization variables include the selection/activation of a subset of participating batteries and the amount of flexibility that should be extracted. Furthermore, the optimization objective corresponds to the expected forecast revenues that can be generated by trading this flexibility to the Day-ahead electricity market. Given the high computationally complexity of a full scale optimization in the case of a large number of participating batteries, we propose a reinforcement-learning-based methodology, which admits linear complexity with the number of participating batteries. The proposed methodology advances prior work with respect to the integration of a large number of batteries. Furthermore, it extends prior work of the authors with respect to providing analytical performance guarantees in comparison with the baseline/nominal operation of the battery. Finally, we compare through simulations the performance of the proposed method with a Linear-programming-based optimization that provides the exact optimum. Copyright (C) 2020 The Authors.

关键词： Control of renewable energy resources Smart grids approximate dynamic programming Day-ahead spot electricity market

来源：评论

学校读者我要写书评

暂无评论

Learning Convex Optimization Control Policies 2

Learning Convex Optimization Control Policies

引用

2nd Annual Conference on Learning for dynamics and Control (L4DC)

作者： Agrawal, Akshay Barratt, Shane Boyd, Stephen Stellato, Bartolomeo 450 Serra Mall Stanford CA 94305 USA 77 Massachusetts Ave Cambridge MA 02139 USA

Many control policies used in applications compute the input or action by solving a convex optimization problem that depends on the current state and some parameters. Common examples of such convex optimization control policies (COCPs) include the linear quadratic regulator (LQR), convex model predictive control (MPC), and convex approximate dynamic programming (ADP) policies. These types of control policies are tuned by varying the parameters in the optimization problem, such as the LQR weights, to obtain good performance, judged by application-specific metrics. Tuning is often done by hand, or by simple methods such as a grid search. In this paper we propose a method to automate this process, by adjusting the parameters using an approximate gradient of the performance metric with respect to the parameters. Our method relies on recently developed methods that can efficiently evaluate the derivative of the solution of a convex program with respect to its parameters. A longer version of this paper, which illustrates our method on many examples, is available at https://***/similar to boyd/papers/learning_***.

关键词： Stochastic control convex optimization approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Ecosystem based multi-species management using Empirical dynamic programming

引用

ECOLOGICAL MODELLING 2021年 441卷

作者： Brias, Antoine Munch, Stephan B. NOAA Southwest Fisheries Sci Ctr Natl Marine Fisheries Serv Santa Cruz CA 95060 USA

Control theory and stochastic dynamic programming have long been used to develop optimal single-species management policies. However, most species interact with others through competition and predation as parts of complex ecosystems. As a consequence, it is unclear how far from optimal the single species policies currently in use actually are. Moreover, there are as yet no scalable algorithms for optimal ecosystem management. Here, we merge recently developed tools from machine learning and nonlinear dynamics to construct and evaluate near-optimal policies in multi-species systems. Specifically, a non-parametric model for the dynamics is estimated from time series data using Gaussian process-based dynamic modeling. A policy is then derived from the inferred dynamics using a temporal difference learning algorithm. Policy performance is benchmarked against single-species policies and the ad hoc ecosystem policies that have been previously offered. We found that EDP policies are closer to the true optimal policies than single-species policies in multi-species systems with two controls and three objectives. The Pareto fronts illustrate the flexibility of EDP policies compared with single-species policies.

关键词： Multi-species management Multi-objectives management Non-linear methods approximate dynamic programming Gaussian process regression Temporal difference learning

来源：评论

学校读者我要写书评

暂无评论

A resilience-based framework for decision making based on simulation-optimization approach

引用

STRUCTURAL SAFETY 2021年 89卷

作者： Nozhati, Saeed Univ Calif Los Angeles B John Garrick Inst Risk Sci 404 Westwood Plaza Los Angeles CA 90095 USA

The functioning of infrastructure systems and buildings in the aftermath of a disaster is critical to the modern urban center. A system study of interdependent networks response and understanding the process of post-disaster recovery is the corner stone of an effective emergency management plan. Exposure to stressors such as chaotic circumstances, budgetary restrictions, time limitations, resource constraints, high-level uncertainty and complexity in the recovery process of a community, highlighting the necessity for comprehensive decision- making framework in the aftermath of disasters. A decision-making framework in the community-level needs to puzzle out large-scale stochastic scheduling problems. These types of problems are difficult stochastic control problems with large combinatorial decision spaces. This study introduces the use of approximate dynamic programming along with heuristics for the determination of optimal recovery actions. The proposed approach prevails the curse of dimensionality and manages multi-state, large-scale infrastructure systems. Further, the proposed approach is able to consider the current recovery policies of responsible companies and improves them along with limited resources. A testbed community coarsely modeled after Gilroy, California, is presented as an illustrative example to show how the proposed approach can be implemented efficiently and accurately to find the optimal decisions. The applied approach provides optimal policies for Electrical Power Network (EPN) and Water Network (WN) of Gilroy in the aftermath of a simulated severe earthquake. The methodology is well suited for any existing infrastructure system with potential extensions to any hazard.

关键词： approximate dynamic programming Community resilience Post-disaster recovery

来源：评论

学校读者我要写书评

暂无评论

Optimal operation framework of customer-premise battery storage for energy charge reduction and primary frequency regulation

引用

JOURNAL OF ENERGY STORAGE 2021年 43卷

作者： Wen, Kerui Li, Weidong Zhao, Yunli Shen, Jiakai Li, Zifeng Zhang, Mingze Dalian Univ Technol Fac Elect Informat & Elect Engn Dalian 116024 Peoples R China Dalian Ocean Univ Sch Informat Engn Dalian 116023 Peoples R China

This paper focuses on the intra-day operation of customer-premise battery storage for multiple applications, in particular energy charge reduction and primary frequency regulation (PFR). The nature of this problem requires the battery to provide reliable PFR, as well as coordinate online optimization performance with computing time while accounting for multidimensional stochastic information. Therefore, we propose a novel optimal operation framework. In the day-ahead phase, the piecewise linear functions (PLFs) are trained offline to approximate the expected operating cost of the long-term future. During the intra-day operation, a two-stage approximate dynamic programming (ADP) model is formulated utilizing the updated short-term future information and the PLFs based long-term estimation. To ensure the reliability of frequency regulation, we develop a robust correction strategy of PFR signals. Combining the correction feedback, the two-stage ADP model is optimized through rolling horizon procedures to dynamically adjust the power basepoint. Numerical simulations demonstrate the performance of our proposed operation framework. The convergence rate of PLFs offline training is 97.3%. In addition, the intra-day operating cost of our proposed framework is merely 1.62% higher than the theoretical optimum, while the online calculation takes only 3.25 s for one rolling period.

关键词： Primary frequency regulation Battery storage Optimal operation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Hourly occupant clothing decisions in residential HVAC energy management

引用

JOURNAL OF BUILDING ENGINEERING 2021年 40卷

作者： Liu, Xuebo Wu, Yingying Zhang, Haifeng Wu, Hongyu Kansas State Univ Mike Wiegers Dept Elect & Comp Engn Manhattan KS 66506 USA Kansas State Univ Dept Interior Design & Fash Studies Manhattan KS 66506 USA Shanghai Univ Engn Sci Sch Mech & Automot Engn Shanghai 201620 Peoples R China

The state-of-the-art energy management for the heating, ventilation and air conditioning (HVAC) system uses a static clothing model that calculates the occupant's clothing insulation as a fixed value based on outdoor air temperature measured at a particular time of the day. However, the static clothing model can hardly capture the occupant's intra-day clothing behaviors, leading to inaccurate thermal comfort assessment and unrealistic HVAC energy management. This paper proposes a novel HVAC energy management scheme to optimally schedule the thermostat setpoints of HVAC and to provide recommendations on occupants' optimal hourly clothing decisions through a predicted mean vote model, while considering uncertainties in the outside temperature. The proposed HVAC energy management scheme is solved by applying an approximate dynamic programming approach. Further, a model predictive control framework with a long short-term memory based forecaster is developed for more realistic simulations. We study the HVAC schedules in a residential home with summer and winter time of use electricity tariffs for both male and female occupants. Compared with non-optimized cases, proof-of-concept simulation results demonstrate that the proposed scheme can achieve a 53.8% and a 29.8% cost saving in a summer-male scenario and a winter-female scenario, respectively.

关键词： Home energy management Occupant's clothing condition approximate dynamic programming Model predictive control Heating Ventilation and air conditioning

来源：评论

学校读者我要写书评

暂无评论

Understanding Consumer dynamic Decision Making Under Competing Loyalty Programs

引用

JOURNAL OF MARKETING RESEARCH 2020年第3期57卷 422-444页

作者： Liu, Jia Ansari, Asim Hong Kong Univ Sci & Technol Mkt Hong Kong Peoples R China Columbia Univ Columbia Business Sch Mkt Div Mkt New York NY 10027 USA

The authors develop an incentive-aligned experimental paradigm to study how consumer purchase dynamics are affected by the interplay between competing firms' loyalty programs and their pricing and promotional strategies. In this experiment, participants made sequential choices between two competing airlines in a stylized frequent traveler task for which an optimal dynamic decision policy can be numerically computed. The authors find that, on average, participants are able to partially realize the long-term benefits from loyalty programs, though most are sensitive to price. They also find that participants' preferences and levels of bounded rationality depend on the nature of the competitive environment, the particular state of each decision scenario, and the type of optimal action. Accordingly, the authors use an approximate dynamic programming model to incorporate boundedly rational decision making. The model classifies participants into five segments that exhibit variation in their performance and decision strategies. Importantly, they find that participants are able to adapt their decision strategies to the environment they face, and thus the overall market outcome and the performance of each firm are influenced by both the competitive environment and the assumption on the extent of consumer optimality.

关键词： approximate dynamic programming bounded rationality competition experiments loyalty programs promotions

来源：评论

学校读者我要写书评

暂无评论

Continuous-time finite-horizon ADP for automated vehicle controller design with high efficiency 3

Continuous-time finite-horizon ADP for automated vehicle con...

引用

3rd International Conference on Unmanned Systems (ICUS)

作者： Lin, Ziyu Duan, Jingliang Li, Shengbo Eben Ma, Haitong Yin, Yuming Cheng, Bo Tsinghua Univ Sch Vehicle & Mobil Beijing Peoples R China

ISBN: (纸本)9781728180250

Automated vehicle controller's design can be formulated into a general optimal control problem. Existing control methods can not meet the millisecond-level time requirements of onboard standard controllers, especially for nonlinear dynamics with non-affine and saturated controller. This paper presents a continuous-time (CT) finite-horizon approximate dynamic programming (ADP) method, which can synthesis off-line nearoptimal control policy with analytical vehicle dynamics. Firstly, we develop a finite-horizon variant of the CT Hamilton-Jacobi-Bellman (HJB) equation. To find the nearly optimal solution of the finite-horizon HJB, i.e., the nearly optimal policy, both value function and policy are approximated by neural networks to map the system states to cost-to-go and control inputs respectively. Then the CT finite-horizon ADP algorithm is proposed based on the generalized policy iteration framework to gradually approach the optimal solution of the finite-horizon HJB. The proposed algorithm can converge to the nearly optimal policy and value function. Finally, we apply the proposed algorithm to the automated vehicle trajectory tracking control in both linear and nonlinear cases. The simulation results demonstrate that the proposed finite-horizon ADP algorithm can obtain the nearly optimal control policy and high online operation efficiency (500 times faster than the nonlinear ipopt solver).

关键词： Automated Vehicle Control approximate dynamic programming Continuous-time Systems Finite-horizon HJB

来源：评论

学校读者我要写书评

暂无评论

Learning nonlinear robust control as a data-driven zero-sum two-player game for an active suspension system 21st

Learning nonlinear robust control as a data-driven zero-sum ...

引用

21st IFAC World Congress on Automatic Control - Meeting Societal Challenges

作者： Radac, Mircea-Bogdan Lala, Timotei Politehn Univ Timisoara Dept Automat & Appl Informat Bd V Parvan 2 RO-300223 Timisoara Romania

An optimal robust control data-driven learning solution is proposed for an active suspension system. The problem is formulated as a zero-sum two-player differential game (ZS-TP-DG), where the optimal control law and the worst-case disturbance control law must be searched for. The distinctive features of the proposed solution are: a Q-learning-like data-driven model-free (with unknown process dynamics) algorithm relying on collected input-state data from the process;neural networks being used as generic function approximators;validation on an active suspension system that is easily amenable to artificial road profile disturbance generation. The superiority of the ZS-TP-DG controller over another optimal controller learned in a disturbance-free context is validated and proven. Copyright (C) 2020 The Authors.

关键词： active suspension system hydraulic actuator neural networks optimal control approximate dynamic programming reinforcement learning two-player zero-sum games

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：