检索结果-内蒙古大学图书馆

Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2013年第1期24卷 145-157页

作者： Heydari, Ali Balakrishnan, Sivasubramanya N. Missouri Univ Sci & Technol Dept Mech & Aerosp Engn Rolla MO 65401 USA

To synthesize fixed-final-time control-constrained optimal controllers for discrete-time nonlinear control-affine systems, a single neural network (NN)-based controller called the Finite-horizon Single Network Adaptive Critic is developed in this paper. Inputs to the NN are the current system states and the time-to-go, and the network outputs are the costates that are used to compute optimal feedback control. Control constraints are handled through a nonquadratic cost function. Convergence proofs of: 1) the reinforcement learning-based training method to the optimal solution;2) the training error;and 3) the network weights are provided. The resulting controller is shown to solve the associated time-varying Hamilton-Jacobi-Bellman equation and provide the fixed-final-time optimal solution. Performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite-time attitude maneuver subject to control bounds. The new formulation has great potential for implementation since it consists of only one NN with single set of weights and it provides comprehensive feedback solutions online, though it is trained offline.

关键词： approximate dynamic programming finite-horizon optimal control fixed-final-time optimal control input-constraint neural networks

来源：评论

学校读者我要写书评

暂无评论

Real-time dispatch management of shared autonomous vehicles with on-demand and pre-booked requests

引用

TRANSPORTATION RESEARCH PART A-POLICY AND PRACTICE 2024年 181卷

作者： Chen, Yao Liu, Yang Baia, Yun Mao, Baohua Beijing Jiaotong Univ Key Lab Transport Ind Big Data Applicat Technol Co Beijing Peoples R China Natl Univ Singapore Dept Civil & Environm Engn Singapore Singapore Natl Univ Singapore Dept Ind Syst Engn & Management Singapore Singapore

Autonomous vehicle technology is poised to revolutionize shared vehicle systems, offering the potential for increased efficiency and convenience. To better devise management strategies for shared autonomous vehicles, this paper addresses a real-time dispatch problem with hybrid requests, where on -demand (immediate) and pre -booked (reserved) trip requests coexist. The coexistence of these two types of request behaviors introduces considerable complexity to realtime dispatch due to the uncertainty in trip demand. We design an approximate dynamic programming (ADP) approach for making vehicle-trip assignments and vehicle relocation decisions. We first formulate the real-time vehicle dispatch problem as a dynamic program and decompose it into time -staged subproblems. To effectively handle the high -dimensional state space, we replace the value functions with tractable approximations and propose a piecewiselinear functional approximation method that captures the spatiotemporal value of vehicles. To calibrate the parameters in the approximations, we propose DualT and DualNext algorithms to provide precise dual information, thereby enhancing the accuracy of our approach. Furthermore, we propose a lookahead strategy that incorporates pre -booked request information into the ADP approach for improving real-time decision -making. We validate the effectiveness of the ADP approach through numerical experiments conducted using taxi data from Brooklyn, New York. The ADP approach outperforms benchmark policies in solution quality while maintaining computational efficiency, and the incorporation of the lookahead strategy significantly enhances the performance of the ADP approach, yielding substantial improvements. Numerical results demonstrate that integrating pre -booked requests into vehicle dispatch management can greatly enhance the system efficiency.

关键词： Shared autonomous vehicles dynamic fleet management approximate dynamic programming Vehicle dispatch Reservation Lookahead

来源：评论

学校读者我要写书评

暂无评论

approximated multi-agent fitted Q iteration

引用

SYSTEMS & CONTROL LETTERS 2023年第1期177卷

作者： Lesage-Landry, Antoine Callaway, Duncan S. Polytech Montreal Dept Elect Engn Mila & GERAD 2500 Polytech Rd Montreal PQ H3T 1J4 Canada Univ Calif Berkeley Energy & Resources Grp 337 Giannini Hall Berkeley CA 94720 USA

We formulate an efficient approximation for multi-agent batch reinforcement learning, the approxi-mated multi-agent fitted Q iteration (AMAFQI). We present a detailed derivation of our approach. We propose an iterative policy search and show that it yields a greedy policy with respect to multiple approximations of the centralized, learned Q-function. In each iteration and policy evaluation, AMAFQI requires a number of computations that scales linearly with the number of agents whereas the analogous number of computations increase exponentially for the fitted Q iteration (FQI), a commonly used approaches in batch reinforcement learning. This property of AMAFQI is fundamental for the design of a tractable multi-agent approach. We evaluate the performance of AMAFQI and compare it to FQI in numerical simulations. The simulations illustrate the significant computation time reduction when using AMAFQI instead of FQI in multi-agent problems and corroborate the similar performance of both approaches. & COPY;2023 Elsevier B.V. All rights reserved.

关键词： approximate dynamic programming Batch reinforcement learning Markov decision process Multi-agent reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

An approximated dynamic programming model for the supply vessel fleet sizing problem

引用

COMPUTERS & OPERATIONS RESEARCH 2024年 163卷

作者： Ribeiro, Leonardo Souza Vistex GmbH Altrottstr 31 D-69190 Walldorf Germany Petrobras SA Ave Chile 65 Rio De Janeiro Brazil Sickingerweg 20 D-69190 Walldorf Germany

The offshore oil and gas exploration and production activities heavily rely on a fleet of supply vessels, which play a crucial role in regularly providing necessary provisions from an onshore supply depot to the offshore installations. For Petrobras, the decision-making process concerning the number of supply vessels to be hired for the upcoming two years holds significant economic implications, given that the daily rates for a single supply vessel can soar to tens of thousands of *** this study, we address the challenges of this heterogeneous fleet sizing problem by presenting an innovative approximated dynamic programming (ADP) model. The model takes into account the uncertainties associated with sea conditions and future demand, making it well-suited for real-world applications. To tackle this problem, we explored two types of approximate value functions: a piecewise linear concave value function and a neural network value function. Both methods underwent rigorous testing and demonstrated their capability to effectively address real-world problems. Consequently, our proposed method exhibits promising potential for practical implementation, allowing companies like Petrobras to accurately assess and determine the ideal fleet composition. By introducing this ADP model, we aim to provide valuable insights and solutions that enhance decision-making processes related to supply vessel fleet sizing in offshore operations. Through considering various uncertainties and complexities, our approach offers a robust and reliable framework to support critical economic decisions in this dynamic industry.

关键词： Oil and gas Logistics Fleet size approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Technician routing and scheduling with employees' learning through implicit cross-training strategy

引用

INTERNATIONAL JOURNAL OF PRODUCTION ECONOMICS 2024年 271卷

作者： Chen, Xi Li, Kaiwen Lin, Sidian Ding, Xiaosong Beijing Foreign Studies Univ Int Business Sch Beijing 100089 Peoples R China Harvard Univ Grad Sch Arts & Sci Cambridge MA 02138 USA Univ Iowa Tippie Coll Business Dept Management Sci Iowa City IA 52242 USA Stockholm Univ Dept Comp & Syst Sci DECIDE SE-16407 Kista Sweden

With record high talent shortages and skill mismatches around the world, this paper investigates a variant of multi -period dynamic technician and routing problem that can be modeled as a Markov decision process. To deal with the double tradeoffs between the routing and service time costs, as well as the current and future costs, we propose an approximate dynamic programming (ADP) -based cost function approximation (CFA) algorithm - the implicit cross -training strategy (ICT). A two-phase routing and scheduling heuristic is developed to account for both employees' learning and future information, and to facilitate an efficient implementation of CFA. Extensive computational results show that ICT can provide a better solution in the current decision with a global view in comparison with the myopic strategy. In depth analysis demonstrates that ICT trains the workforce with more balanced skillsets and workloads, which ensures the flexibility of the workforce and helps buffer against the future uncertainties with substantial routing cost savings. Additionally, ICT has much more advantages in large-scale problems with more diversified service requests and randomly distributed customers.

关键词： Routing and scheduling approximate dynamic programming Learning Cost function approximation Workforce management

来源：评论

学校读者我要写书评

暂无评论

Optimal operation framework of customer-premise battery storage for energy charge reduction and primary frequency regulation

引用

JOURNAL OF ENERGY STORAGE 2021年 43卷

作者： Wen, Kerui Li, Weidong Zhao, Yunli Shen, Jiakai Li, Zifeng Zhang, Mingze Dalian Univ Technol Fac Elect Informat & Elect Engn Dalian 116024 Peoples R China Dalian Ocean Univ Sch Informat Engn Dalian 116023 Peoples R China

This paper focuses on the intra-day operation of customer-premise battery storage for multiple applications, in particular energy charge reduction and primary frequency regulation (PFR). The nature of this problem requires the battery to provide reliable PFR, as well as coordinate online optimization performance with computing time while accounting for multidimensional stochastic information. Therefore, we propose a novel optimal operation framework. In the day-ahead phase, the piecewise linear functions (PLFs) are trained offline to approximate the expected operating cost of the long-term future. During the intra-day operation, a two-stage approximate dynamic programming (ADP) model is formulated utilizing the updated short-term future information and the PLFs based long-term estimation. To ensure the reliability of frequency regulation, we develop a robust correction strategy of PFR signals. Combining the correction feedback, the two-stage ADP model is optimized through rolling horizon procedures to dynamically adjust the power basepoint. Numerical simulations demonstrate the performance of our proposed operation framework. The convergence rate of PLFs offline training is 97.3%. In addition, the intra-day operating cost of our proposed framework is merely 1.62% higher than the theoretical optimum, while the online calculation takes only 3.25 s for one rolling period.

关键词： Primary frequency regulation Battery storage Optimal operation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Hourly occupant clothing decisions in residential HVAC energy management

引用

JOURNAL OF BUILDING ENGINEERING 2021年 40卷

作者： Liu, Xuebo Wu, Yingying Zhang, Haifeng Wu, Hongyu Kansas State Univ Mike Wiegers Dept Elect & Comp Engn Manhattan KS 66506 USA Kansas State Univ Dept Interior Design & Fash Studies Manhattan KS 66506 USA Shanghai Univ Engn Sci Sch Mech & Automot Engn Shanghai 201620 Peoples R China

The state-of-the-art energy management for the heating, ventilation and air conditioning (HVAC) system uses a static clothing model that calculates the occupant's clothing insulation as a fixed value based on outdoor air temperature measured at a particular time of the day. However, the static clothing model can hardly capture the occupant's intra-day clothing behaviors, leading to inaccurate thermal comfort assessment and unrealistic HVAC energy management. This paper proposes a novel HVAC energy management scheme to optimally schedule the thermostat setpoints of HVAC and to provide recommendations on occupants' optimal hourly clothing decisions through a predicted mean vote model, while considering uncertainties in the outside temperature. The proposed HVAC energy management scheme is solved by applying an approximate dynamic programming approach. Further, a model predictive control framework with a long short-term memory based forecaster is developed for more realistic simulations. We study the HVAC schedules in a residential home with summer and winter time of use electricity tariffs for both male and female occupants. Compared with non-optimized cases, proof-of-concept simulation results demonstrate that the proposed scheme can achieve a 53.8% and a 29.8% cost saving in a summer-male scenario and a winter-female scenario, respectively.

关键词： Home energy management Occupant's clothing condition approximate dynamic programming Model predictive control Heating Ventilation and air conditioning

来源：评论

学校读者我要写书评

暂无评论

Convergence analysis of the deep neural networks based globalized dual heuristic programming

引用

AUTOMATICA 2020年 122卷 109222-109222页

作者： Kim, Jong Woo Oh, Tae Hoon Son, Sang Hwan Jeong, Dong Hwi Lee, Jong Min Seoul Natl Univ Engn Dev Res Ctr 1 Gwanak Ro Seoul 08826 South Korea Seoul Natl Univ Inst Chem Proc Sch Chem & Biol Engn 1 Gwanak Ro Seoul 08826 South Korea Texas A&M Univ Artie McFerrin Dept Chem Engn College Stn TX 77845 USA

Globalized dual heuristic programming (GDHP) algorithm is a special form of approximate dynamic programming (ADP) method that solves the Hamilton-Jacobi-Bellman (HJB) equation for the case where the system takes control-affine form subject to the quadratic cost function. This study incorporates the deep neural networks (DNNs) as a function approximator to inherit the advantages of which to express high-dimensional function space. Elementwise error bound of the costate function sequence is newly derived and the convergence property is presented. In the approximated function space, uniformly ultimate boundedness (UUB) condition for the weights of the general multi-layer NNs weights is obtained. It is also proved that under the gradient descent method for solving the moving target regression problem, UUB gradually converges to the value, which exclusively contains the approximation reconstruction error. The proposed method is demonstrated on the continuous reactor control in aims to obtain the control policy for multiple initial states, which justifies the necessity of DNNs structure for such cases. (c) 2020 Elsevier Ltd. All rights reserved.

关键词： approximate dynamic programming Reinforcement learning Deep neural networks Lyapunov stability Nonlinear control

来源：评论

学校读者我要写书评

暂无评论

Ecosystem based multi-species management using Empirical dynamic programming

引用

ECOLOGICAL MODELLING 2021年 441卷 109423-109423页

作者： Brias, Antoine Munch, Stephan B. NOAA Southwest Fisheries Sci Ctr Natl Marine Fisheries Serv Santa Cruz CA 95060 USA

Control theory and stochastic dynamic programming have long been used to develop optimal single-species management policies. However, most species interact with others through competition and predation as parts of complex ecosystems. As a consequence, it is unclear how far from optimal the single species policies currently in use actually are. Moreover, there are as yet no scalable algorithms for optimal ecosystem management. Here, we merge recently developed tools from machine learning and nonlinear dynamics to construct and evaluate near-optimal policies in multi-species systems. Specifically, a non-parametric model for the dynamics is estimated from time series data using Gaussian process-based dynamic modeling. A policy is then derived from the inferred dynamics using a temporal difference learning algorithm. Policy performance is benchmarked against single-species policies and the ad hoc ecosystem policies that have been previously offered. We found that EDP policies are closer to the true optimal policies than single-species policies in multi-species systems with two controls and three objectives. The Pareto fronts illustrate the flexibility of EDP policies compared with single-species policies.

关键词： Multi-species management Multi-objectives management Non-linear methods approximate dynamic programming Gaussian process regression Temporal difference learning

来源：评论

学校读者我要写书评

暂无评论

A resilience-based framework for decision making based on simulation-optimization approach

引用

STRUCTURAL SAFETY 2021年 89卷 102032-102032页

作者： Nozhati, Saeed Univ Calif Los Angeles B John Garrick Inst Risk Sci 404 Westwood Plaza Los Angeles CA 90095 USA

The functioning of infrastructure systems and buildings in the aftermath of a disaster is critical to the modern urban center. A system study of interdependent networks response and understanding the process of post-disaster recovery is the corner stone of an effective emergency management plan. Exposure to stressors such as chaotic circumstances, budgetary restrictions, time limitations, resource constraints, high-level uncertainty and complexity in the recovery process of a community, highlighting the necessity for comprehensive decision- making framework in the aftermath of disasters. A decision-making framework in the community-level needs to puzzle out large-scale stochastic scheduling problems. These types of problems are difficult stochastic control problems with large combinatorial decision spaces. This study introduces the use of approximate dynamic programming along with heuristics for the determination of optimal recovery actions. The proposed approach prevails the curse of dimensionality and manages multi-state, large-scale infrastructure systems. Further, the proposed approach is able to consider the current recovery policies of responsible companies and improves them along with limited resources. A testbed community coarsely modeled after Gilroy, California, is presented as an illustrative example to show how the proposed approach can be implemented efficiently and accurately to find the optimal decisions. The applied approach provides optimal policies for Electrical Power Network (EPN) and Water Network (WN) of Gilroy in the aftermath of a simulated severe earthquake. The methodology is well suited for any existing infrastructure system with potential extensions to any hazard.

关键词： approximate dynamic programming Community resilience Post-disaster recovery

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：