检索结果-内蒙古大学图书馆

Transmission scheduling for multi-process multi-sensor remote estimation via approximate dynamic programming

AUTOMATICA 2022年 136卷 110061-110061页

作者： Forootani, Ali Iervolino, Raffaele Tipaldi, Massimo Dey, Subhrakanti Maynooth Univ Hamilton Inst Maynooth W23 F2K8 Kildare Ireland Univ Naples Federico II Dept Elect Engn & Informat Technol I-80125 Naples Italy Univ Sannio Dept Engn I-82100 Benevento Italy

In this paper, we consider a remote estimation problem where multiple dynamical systems are observed by smart sensors, which transmit their local estimates to a remote estimator over channels prone to packet losses. Unlike previous works, we allow multiple sensors to transmit simultaneously even though they can cause interference, thanks to the multi-packet reception capability at the remote estimator. In this setting, the remote estimator can decode multiple sensor transmissions (successful packet arrivals) as long as their signal-to-interference-and-noise ratios (SINR) are above a certain threshold. In this setting, we address the problem of optimal sensor transmission scheduling by minimizing a finite horizon discounted expected estimation error covariance cost across all systems at the remote estimator, subject to an average transmission cost. While this problem can be posed as a stochastic control problem, the optimal solution requires solving a Bellman equation for a dynamic programming (DP) problem, the complexity of which scales exponentially with the number of systems being measured and their state dimensions. In this paper, we resort to a novel Least Squares Temporal Difference (LSTD) approximate dynamic programming (ADP) based approach to approximating the value function. More specifically, an off-policy based LSTD approach, named in short Enhanced Exploration Greedy LSTD (EG-LSTD), is proposed. We discuss the convergence analysis of the EG-LSTD algorithm and its implementation. A Python based program is developed to implement and analyse the different aspects of the proposed method. Simulation examples are presented to support the results of the proposed approach both for the exact DP and ADP cases. (C) 2021 Elsevier Ltd. All rights reserved.

关键词： Markov Decision Process approximate dynamic programming Least Squares Temporal Difference Wireless sensor networks Kalman filter Sensor scheduling

来源：评论

学校读者我要写书评

暂无评论

Risk-averse real-time dispatch of integrated electricity and heat system using a modified approximate dynamic programming approach

引用

ENERGY 2020年 198卷 117347-000页

作者： Pan, Zhenning Yu, Tao Li, Jie Qu, Kaiping Yang, Bo South China Univ Technol Coll Elect Power Guangzhou 510640 Peoples R China Kunming Univ Sci & Technol Fac Elect Power Engn Kunming 650500 Yunnan Peoples R China

Coordinated operation of integrated electricity and heat system can improve operation flexibility and reduce cost. However, multiple uncertainties challenge its optimal operation. This paper aims at developing a risk-averse and computationally efficient policy for real-time stochastic dispatch of integrated electricity and heat system, which improves the economy as well as avoiding the risk of high costs in critical scenarios. First, real-time dispatch of integrated electricity and heat system is formulated as a multistage risk-averse stochastic sequential optimization problem with dynamic risk measure, where combined heat and power unit, energy storage, flexible electricity and heat load are jointly utilized to minimize the risk-adjusted total costs. Next, a risk-averse dynamic programming formulation of the original problem is presented, upon which a data-driven risk-averse approximate dynamic programming is employed to address computational challenge, and develop almost optimal and computationally efficient policy. By exploiting information from training samples in off-line learning, the proposed algorithm can efficiently responses to the stochastic exogenous information. Comparative simulations with different risk-aversion preferences and different methods verify the effectiveness of the proposed algorithm. (C) 2020 Elsevier Ltd. All rights reserved.

关键词： Integrated electricity and heat system Real-time optimization Risk-averse optimization Stochastic optimization approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

An approximate dynamic programming approach to the admission control of elective patients

引用

COMPUTERS & OPERATIONS RESEARCH 2021年 132卷 105259-105259页

作者： Zhang, Jian Dridi, Mahjoub El Moudni, Abdellah Eindhoven Univ Technol Sch Ind Engn NL-5600 MB Eindhoven Netherlands Univ Bourgogne Franche Comte Lab Nanomed Imagerie & Therapeut UTBM Rue Thierry Mieg F-90010 Belfort France

In this paper, we propose an approximate dynamic programming (ADP) algorithm to solve a Markov decision process (MDP) formulation for the admission control of elective patients. To manage the elective patients from multiple specialties equitably and efficiently, we establish a waiting list and assign each patient a time-dependent dynamic priority score. Then, taking the random arrivals of patients into account, sequential decisions are made on a weekly basis. At the end of each week, we select the patients to be treated in the following week from the waiting list. By minimizing the cost function of the MDP over an infinite horizon, we seek to achieve the best trade-off between the patients' waiting times and the over-utilization of surgical resources. Considering the curses of dimensionality resulting from the large scale of realistically sized problems, we first analyze the structural properties of the MDP and propose an algorithm that facilitates the search for best actions. We then develop a novel reinforcement-learning-based ADP algorithm as the solution technique. Experimental results reveal that the proposed algorithms consume much less computation time in comparison with that required by conventional dynamic programming methods. Additionally, the algorithms are shown to be capable of computing high-quality near-optimal policies for realistically sized problems.

关键词： Patient admission control Operating theatre planning Markov decision process approximate dynamic programming Recursive least-squares temporal difference learning

来源：评论

学校读者我要写书评

暂无评论

New closed-loop approximate dynamic programming for solving stochastic decentralized multi-project scheduling problem with resource transfers

引用

EXPERT SYSTEMS WITH APPLICATIONS 2021年 185卷 115593-115593页

作者： Zhao, Song Xu, Zhe Beihang Univ Sch Econ & Management Beijing 100191 Peoples R China

Resource transfers and uncertain activity durations are two practical factors that are difficult to cope with in real project management. This paper studies a new decentralized multi-project scheduling problem with both resource transfers and uncertain activity durations, we name it the stochastic decentralized multi-project scheduling problem with resource transfers (SDRCMPSPTT). To tackle the curse-of-dimensionality of an exact solution approach, we develop a new and effective rollout policy-based approximate dynamic programming (ADP) algorithm for SDRCMPSPTT. Based on the benchmark instances, we build a new data set and examine the performance of 12 priority rule heuristics, then select the two best ones as the base policy for our rollout algorithm. The proposed approach is verified under a stochastic environment by assessing different base heuristic policies, computational results show that the rollout algorithm can further improve the solution quality of the base heuristics. Compared with the state-of-the-art algorithm for solving the decentralized multi project scheduling problem under a deterministic environment, our rollout policy-based ADP algorithm can also obtain competitive solutions.

关键词： Decentralized multi-project scheduling Resource transfers Uncertain activity durations approximate dynamic programming Rollout algorithm

来源：评论

学校读者我要写书评

暂无评论

Solving optimal predictor-feedback control using approximate dynamic programming

引用

AUTOMATICA 2024年 170卷

作者： Wang, Hongxia Zhao, Fuyu Zhang, Zhaorong Xu, Juanjuan Li, Xun Shandong Univ Sci & Technol Sch Elect Engn & Automat Qingdao Peoples R China Shandong Univ Sch Comp Sci & Technol Qingdao Peoples R China Shandong Univ Sch Control Sci & Engn Jinan Peoples R China Hong Kong Polytech Univ Dept Appl Math Hong Kong Peoples R China

This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati-ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati-ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm. (c) 2024 Published by Elsevier Ltd.

关键词： Stochastic system Optimal control Input delay approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Large-scale dynamic system optimization using dual decomposition method with approximate dynamic programming

引用

SYSTEMS & CONTROL LETTERS 2021年 150卷 104894-104894页

作者： Rokhforoz, Pegah Kebriaei, Hamed Ahmadabadi, Majid Nili Univ Tehran Sch Elect & Comp Engn Coll Engn Tehran Iran Inst Res Fundamental Sci IPM Sch Comp Sci POB 19395-5746 Tehran Iran

In this paper, multi-agent dynamic optimization with a coupling constraint is studied. The aim is to minimize a strongly convex social cost function, by considering a linear stochastic dynamics for each agent and also coupling constraints among the agents. In order to handle the coupling constraint and also, to avoid high computational cost imposed by a centralized method for large scale systems, the dual decomposition method is used to decompose the problem into multiple individual sub-problems, while the dual variable is adjusted by a coordinator. Nevertheless, since each sub-problem is not a linear-quadratic (LQ) optimal control problem, and hence its closed-form solution does not exist, approximate dynamic programming (ADP) is utilized to solve the sub-problems. The main contribution of the paper is to propose an algorithm by considering the interrelated iterations of dual variable adjustment and ADP, and to prove the convergence of the algorithm to the global optimal solution of the social cost function. Additionally, the implementation of the proposed algorithm using a neural network is presented. Also, the computational advantage of the proposed algorithm in comparison with other bench-marking methods is discussed in simulation results. (C) 2021 Elsevier B.V. All rights reserved.

关键词： Multi-agent system approximate dynamic programming Duality theory Coupling constraint

来源：评论

学校读者我要写书评

暂无评论

Research on the remaining life prediction technology of environmentally friendly gas switchgear based on approximate dynamic programming

引用

FRONTIERS IN ENERGY RESEARCH 2024年 12卷

作者： Zhang, Ziyu Ye, Yirong Xue, Lin Tang, Baofeng He, Lei Zhao, Hongtao Zhang, Pengliang Sun, Shengxia Fan, Jie State Grid Xiongan New Area Elect Power Supply Co Xiongan New Area Peoples R China State Grid Fuzhou Elect Power Supply Co Fuzhou Peoples R China State Grid Hebei Elect Power Res Inst Shijiazhuang Peoples R China

The transformation of energy structure has brought about new changes in the power system, and the environmentally friendly gas switchgear with the goal of low carbon and environmental protection has been widely spread and applied, but due to its short application time and the accumulation of related research, it continues to carry out life prediction to improve the related operation and maintenance system. The article proposes the environmental protection gas switchgear data processing technology based on data enhancement technology, proposes the environmental protection gas switchgear life prediction technology based on approximate dynamic planning, and verifies the algorithm through case analysis, which proves the effectiveness and accuracy of the proposed method.

关键词： data-driven environmental gas switch cabinet approximate dynamic programming life prediction data enhancement technology

来源：评论

学校读者我要写书评

暂无评论

Prediction-failure-risk-aware online dial-a-ride scheduling considering spatial demand correlation via approximate dynamic programming and scenario approach

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2024年 169卷

作者： Wu, Weitiao Zou, Honghui Liu, Ronghui South China Univ Technol Sch Civil Engn & Transportat Guangzhou 510641 Peoples R China Univ Leeds Inst Transport Studies Leeds LS2 9JT England

The dial-a-ride (DAR) service is a precursor to emerging shared mobility. Service providers expect efficient management of fleet resources to improve service quality without degrading economic viability. Most existing studies overlook possible future demands that could yield better matching opportunities and scheduling benefits, and therefore have short-sighted limitations. Moreover, the effects of correlated demand and potential prediction errors were ignored. To address these gaps, this paper investigates prediction-failure-risk-aware online DAR scheduling with spatial demand correlation. Request selection and cancellation are explicitly considered. We formulate the problem as a Markov decision process (MDP) and solve it by approximate dynamic programming (ADP). We further develop a demand prediction model that can capture the characteristics of DAR travel demand (uncertainty, sparsity, and spatial correlation). Deep quantile regression is adopted to estimate the marginal distribution of each OD pair. These marginals are combined into a joint demand distribution by constructing a Gaussian Copula to capture the spatial demand correlation. A prediction error correction mechanism is proposed to eliminate prediction errors and rectify policies promptly. Based on the model properties, several families of customized pruning strategies are devised to improve the computational efficiency and solution quality of ADP. We solve policies over time in the dynamic environment mixed with actual and stochastic future demands via the ADP algorithm and scenario approach. We propose the value function rolling method and multi-scenario exploration method, to address the deviation of the value function and identify the optimal policy from multiple future demand scenarios. Numerical results demonstrate the importance and benefits of incorporating demand forecasting and spatial correlation into the DAR operation. The improvement due to prediction is significant even when the prediction

关键词： Dial-a-ride service OD demand prediction Spatial demand correlation approximate dynamic programming Risk-aware decision-making

来源：评论

学校读者我要写书评

暂无评论

Real-time joint regulating reserve deployment of electric vehicles and coal-fired generators considering EV battery degradation using scalable approximate dynamic programming

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2022年 140卷 108017-108017页

作者： Xue, Xizhen Fang, Jiakun Ai, Xiaomeng Cui, Shichang Xu, Mengyao Yao, Wei Wen, Jinyu Huazhong Univ Sci & Technol Sch Elect & Elect Engn State Key Lab Adv Electromagnet Engn & Technol Wuhan 430074 Peoples R China

This paper proposes a real-time joint regulating reserve deployment (RRD) model of electric vehicles (EVs) and coal-fired generators (CGs) considering EV battery degradation. The superiority of the proposed model lies in the integration of the EVs' fast regulation speed and CGs' large regulation capacity. The whole optimization model is formulated under a two-stage Markov Decision Process (MDP) to correspond to the real-time RRD process. The user behavior characteristic and EV battery degradation are taken into account to stimulate the users to participate in RRD. Based on the formulated MDP, a piece-wise linear (PWL) function based scalable approximate dynamic programming (ADP) algorithm for different EV clusters is constructed to solve the realtime RRD model under uncertainties. A separable slope update method is proposed to update the slopes of PLF for different EV clusters with different user behaviors. The proposed ADP based real-time RRD algorithm (ADP-RTRRD) provides the approximate optimal real-time RRD policy with the empirical knowledge embedded through off-line training. Numerical simulation on a modified IEEE-39 bus system and Henan power grid in China verify the superiority of the proposed joint RRD model and the advantages of the proposed ADP-RTRRD algorithm.

关键词： Real-time joint regulating reserve deployment Electric vehicle Piece-wise linear function Separable slope update approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Technical Note-An approximate dynamic programming Approach to the Incremental Knapsack Problem

引用

OPERATIONS RESEARCH 2023年第4期71卷 1414-1433页

作者： Aouad, Ali Segev, Danny London Business Sch Dept Management Sci & Operat London NW1 4SA England Tel Aviv Univ Sch Mathemat Sci Dept Stat & Operat Res IL-69978 Tel Aviv Israel

We study the incremental knapsack problem, where one wishes to sequentially pack items into a knapsack whose capacity expands over a finite planning horizon, with the objective of maximizing time-averaged profits. Although various approximation algorithms were developed under mitigating structural assumptions, obtaining nontrivial performance guarantees for this problem in its utmost generality has remained an open question thus far. In this paper, we devise a polynomial-time approximation scheme for general instances of the incremental knapsack problem, which is the strongest guarantee possible given existing hardness results. In contrast to earlier work, our algorithmic approach exploits an approximate dynamic programming formulation. Starting with a simple exponentially sized dynamic program, we prove that an appropriate composition of state pruning ideas yields a polynomially sized state space with negligible loss of optimality. The analysis of this formulation synthesizes various techniques, including new problem decompositions, parsimonious counting arguments, and efficient rounding methods, that may be of broader interest.

关键词： incremental knapsack approximate dynamic programming PTAS

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：