检索结果-内蒙古大学图书馆

Multi-period orienteering with uncertain adoption likelihood and waiting at customers

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2020年第1期282卷 288-303页

作者： Zhang, Shu Ohlmann, Jeffrey W. Thomas, Barrett W. Chongqing Univ Sch Econ & Business Adm Chongqing Peoples R China Univ Iowa Tippie Coll Business Dept Business Analyt Iowa City IA USA

We introduce a multi-period orienteering problem in which a salesperson visits customers to provide information on product and service offerings that may lead to eventual adoptions at the end of the problem horizon. Upon arriving at a customer, the salesperson may experience uncertain wait time that delays or prevents the salesperson from meeting with the customer within the customer's time window. A customer's adoption probability evolves stochastically from period to period depending on whether the salesperson meets with the customer. To maximize the expected sales accrued from product adoptions at the end of the horizon, the salesperson must decide in which period(s) to visit the customers and what sequence to visit the customers within each period. We formally model this problem as a Markov decision process. To overcome the computational challenges induced by the large state and action space, we propose a two-stage heuristic approach to facilitate decision making. In the first stage, we solve an assignment problem to determine which customers to visit in the current period. In the second stage, we solve a routing problem to determine the sequence to visit the selected customers. In the computational experiments, we demonstrate the effectiveness of our heuristic methods. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Routing Uncertain product adoption Stochastic wait times Multi-period horizon approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Discrete-Time Impulsive Adaptive dynamic programming

引用

IEEE TRANSACTIONS ON CYBERNETICS 2020年第10期50卷 4293-4306页

作者： Wei, Qinglai Song, Ruizhuo Liao, Zehua Li, Benkai Lewis, Frank L. Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Univ Texas Arlington UTA Res Inst Arlington TX 76118 USA Northeastern Univ Shenyang 110036 Peoples R China

In this paper, a new iterative adaptive dynamic programming (ADP) algorithm is developed to solve optimal impulsive control problems for infinite horizon discrete-time nonlinear systems. Considering the constraint of the impulsive interval, in each iteration, the iterative impulsive value function under each possible impulsive interval is obtained, and then the iterative value function and iterative control law are achieved. A new convergence analysis method is developed which proves an iterative value function to converge to the optimum as the iteration index increases to infinity. The properties of the iterative control law are analyzed, and the detailed implementation of the optimal impulsive control law is presented. Finally, two simulation examples with comparisons are given to show the effectiveness of the developed method.

关键词： Optimal control Performance analysis Nonlinear systems dynamic programming Heuristic algorithms Indexes Adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming impulsive control nonlinear systems optimal control

来源：评论

学校读者我要写书评

暂无评论

Real-time scheduling of semi-urgent patients under waiting time targets

引用

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH 2020年第4期58卷 1127-1143页

作者： Wen, Jing Geng, Na Xie, Xiaolan Shanghai Jiao Tong Univ Dept Ind Engn & Management Shanghai Peoples R China Shanghai Jiao Tong Univ SinoUS Global Logist Inst Shanghai Peoples R China CNRS Ctr CIS Mines St Etienne UMR LIMOS 6158 St Etienne France Shanghai Jiao Tong Univ Antai Coll Econ & Management Shanghai Peoples R China

Semi-urgent patients arrive at an emergency department and visit the physician after triage. Patients right after triage should be served within a maximum allowable waiting time;whereas in-process patients need to be served as soon as possible to avoid adverse events. The physician must determine which one to be served next. To deal with this problem, a Markov decision process (MDP) is proposed for real-time scheduling. The wait of patients right after triage incurs a non-decreasing marginal waiting cost in their lateness, whereas the wait of in-process patients incurs linear cost function. The objective is to minimise the total weighted waiting cost. The properties of the MDP model are analysed. In the special case of long examination time and common treatment rate for all patients, we prove the multimodularity of the value function and the optimality of state-dependent threshold policies. Based on these properties, efficient heuristic policies and an approximate dynamic programming (ADP) policy are proposed. A threshold policy, which is defined by the function of expected tardiness of patients right after triage, is found to excel in all experiments, with average gaps less than 0.7% from the optimal control in small-size instances and 0.18% from ADP in real application.

关键词： emergency department real-time scheduling waiting time target Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Emergency medical service resource allocation in a mass casualty incident by integrating patient prioritization and hospital selection problems

引用

IISE TRANSACTIONS 2020年第10期52卷 1141-1155页

作者： Shin, Kyohong Lee, Taesik Korea Adv Inst Sci & Technol Dept Ind & Syst Engn Daejeon South Korea

Mass casualty incidents often cause a shortage of resources for emergency medical services such as ambulances and emergency departments. These resources must be effectively managed to save as many lives as possible. Critical decisions in operating emergency medical service systems include the prioritization of patients for ambulance transport and the selection of destination hospitals. We develop a stochastic dynamic model that integrates patient transport prioritization and hospital selection problems. Policy solutions from the model are compared with other plausible heuristics, and our experimental results show that our policy solution outperforms other alternatives. More importantly, we show that there are considerable benefits from optimally selecting hospitals, which suggests that this decision is just as important as the patient prioritization decision. Motivated by the finding, we propose a heuristic policy that considers both patient prioritization and hospital selection. Experimental results demonstrate strong performance of our heuristic policy compared with existing heuristics. In addition, the proposed approach offers practical advantages. Whereas the existing heuristic policies use patient information, our heuristic policy requires information on the hospital state, which is more readily available and reliable.

关键词： OR in disaster relief emergency medical service mass casualty incident triage patient prioritization disaster management approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Optimal stochastic dynamic scheduling for managing community recovery from natural hazards

引用

RELIABILITY ENGINEERING & SYSTEM SAFETY 2020年 193卷

作者： Nozhati, Saeed Sarkale, Yugandhar Chong, Edwin K. P. Ellingwood, Bruce R. Colorado State Univ Dept Civil & Environm Engn Ft Collins CO 80523 USA Colorado State Univ Dept Elect & Comp Engn Ft Collins CO 80523 USA Colorado State Univ Dept Math Ft Collins CO 80523 USA

Following the occurrence of an extreme natural or man-made event, community recovery management should aim at providing optimal restoration policies for a community over a planning horizon. Calculating such optimal restoration policies in the presence of uncertainty poses significant challenges for community leaders. Stochastic scheduling for several interdependent infrastructure systems is a difficult control problem with huge decision spaces. The Markov decision process (MDP)-based optimization approach proposed in this study incorporates different sources of uncertainties to compute the restoration policies. The computation of optimal scheduling presented herein employs the rollout algorithm, which provides an effective computational tool for optimization problems dealing with real-world large-scale networks and communities. The proposed methodology is applied to a realistic community recovery problem, where different decision-making objectives are considered. The approach accommodates current restoration strategies employed in recovery management;computational results indicate that the restoration policies identified herein significantly outperform the current recovery strategies. Finally, the applicability of the method to address different risk attitudes of policymakers, which include risk-neutral and risk-averse attitudes in the community recovery management, is examined.

关键词： approximate dynamic programming Community-level decision making Community recovery management Markov decision process Optimization Rollout

来源：评论

学校读者我要写书评

暂无评论

Mean Field Game-based Decentralized Optimal Charging Control for large-scale of electric Vehicles

引用

IFAC-PapersOnLine 2022年第15期55卷 111-116页

作者： Zejian Zhou Hao Xu Department of electric & Computer Engineering and Computer Science University of New Haven West Haven CT 06516 USA Department of Electrical & Biomedical Engineering University of Nevada Reno Reno NV 89557 USA

In this paper, the online decentralized optimal charging control problem for large-scale electric vehicles is investigated. Massive concurrent charging, which often happens for the electric buses in a smart city, is devastating to the power grid's supply capacity due to the instant high load demand. Traditional strategies rely on a centralized dispatch center which suffers from the rising computational complexity and the data communication burden when the bus number is huge. In the novel design, a decentralized optimal charging control strategy is developed to tackle these challenges. By engaging the emerging Mean Field Games (MFG) theory with a novel Actor-Critic-Mass (ACM) reinforcement learning scheme, a probability density function (PDF) of the massive state of charge (SOC) is used to encode all electric vehicles’ charging demand information into a fix-dimension function. Moreover, the optimal charging control and the PDF of massive SOC can be solved through the coupled Fokker-Planck-Kolmogorov (FPK) and Hamilton-Jacobi-Bellman (HJB) equations in a decentralized manner. Eventually, the effectiveness of the proposed scheme is validated through a series of numerical simulations.

关键词： Mean Field Games reinforcement learning approximate dynamic programming multi-agent systems reinforcement learning control

来源：评论

学校读者我要写书评

暂无评论

A reinforcement learning approach to the stochastic cutting stock problem

引用

EURO JOURNAL ON COMPUTATIONAL OPTIMIZATION 2022年 10卷

作者： Pitombeira-Neto, Anselmo R. Murta, Arthur H. F. Univ Fed Ceara Dept Ind Engn Campus Pici Fortaleza Brazil Univ Fed Ceara Grad Program Modeling & Quantitat Methods Fortaleza Brazil Univ Fed Ceara OPL Lab Operat Res Prod & Logist Fortaleza Brazil

We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock in anticipation of the unknown demand. An optimal solution corresponds to a policy that associates each state with a decision and minimizes the expected total cost. Since exact algorithms scale exponentially with the state-space dimension, we develop a heuristic solution approach based on reinforcement learning. We propose an approximate policy iteration algorithm in which we apply a linear model to approximate the action-value function of a policy. Policy evaluation is performed by solving the projected Bellman equation from a sample of state transitions, decisions and costs obtained by simulation. Due to the large decision space, policy improvement is performed via the cross-entropy method. Computational experiments are carried out with the use of realistic data to illustrate the application of the algorithm. Heuristic policies obtained with polynomial and Fourier basis functions are compared with myopic and random policies. Results indicate the possibility of obtaining policies capable of adequately controlling inventories with an average cost up to 80% lower than the cost obtained by a myopic policy.(c) 2022 The Author(s). Published by Elsevier Ltd on behalf of Association of European Operational Research Societies (EURO).

关键词： Cutting stock problem Reinforcement learning approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate information state for approximate planning and reinforcement learning in partially observed systems

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2022年第1期23卷 483-565页

作者： Jayakumar Subramanian Amit Sinha Raihan Seraj Aditya Mahajan Media and Data Science Research Lab Digital Experience Cloud Adobe Systems India Private Limited Noida Uttar Pradesh India Department of Electrical and Computer Engineering McGill University Montreal Quebec Canada

We propose a theoretical framework for approximate planning and learning in partially observed systems. Our framework is based on the fundamental notion of information state. We provide two definitions of information state--i) a function of history which is sufficient to compute the expected reward and predict its next value; ii) a function of the history which can be recursively updated and is sufficient to compute the expected reward and predict the next observation. An information state always leads to a dynamic programming decomposition. Our key result is to show that if a function of the history (called approximate information state (AIS)) approximately satisfies the properties of the information state, then there is a corresponding approximate dynamic program. We show that the policy computed using this is approximately optimal with bounded loss of optimality. We show that several approximations in state, observation and action spaces in literature can be viewed as instances of AIS. In some of these cases, we obtain tighter bounds. A salient feature of AIS is that it can be learnt from data. We present AIS based multi-time scale policy gradient algorithms and detailed numerical experiments with low, moderate and high dimensional environments.

关键词： partially observed reinforcement learning partially observable Markov decision processes approximate dynamic programming information state approximate information state

来源：评论

学校读者我要写书评

暂无评论

Pharmaceutical R & D pipeline management under trial duration uncertainty

引用

COMPUTERS & CHEMICAL ENGINEERING 2020年 136卷

作者： Gokalp, Elvan Branke, Juergen Univ Warwick WMG Coventry CV4 7AL W Midlands England Univ Warwick Warwick Business Sch Coventry CV4 7AL W Midlands England

We consider a pharmaceutical Research & Development (R & D) pipeline management problem under two significant uncertainties: the outcomes of clinical trials and their durations. We present an approximate dynamic programming (ADP) approach to solve the problem efficiently. Given an initial list of potential drug candidates, ADP derives a policy that suggests the trials to be performed at each decision point and state. For the classical R&D pipeline planning problem with deterministic trial durations, we compare our ADP approach with other methods from the literature, and find that it can find better solutions more quickly in particular for larger problem instances. For the case with stochastic trial durations, we compare the ADP algorithm with a myopic approach and show that the expected net profit obtained by the derived ADP policy is higher (almost 20% for a 10-drug portfolio). (C) 2020 Elsevier Ltd. All rights reserved.

关键词： dynamic programming Pharmaceutical R&D pipeline management Heuristics approximate dynamic programming Project scheduling

来源：评论

学校读者我要写书评

暂无评论

Distributed approximate dynamic Control for Traffic Management of Busy Railway Networks

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2020年第9期21卷 3788-3798页

作者： Ghasempour, Taha Nicholson, Gemma L. Kirkwood, David Fujiyama, Taku Heydecker, Benjamin UCL Ctr Transport Studies Fac Engn Sci London WC1E 6BT England Univ Birmingham Birmingham Ctr Railway Res & Educ Birmingham B15 2TT W Midlands England

Railway operations are prone to disturbances that can rapidly propagate through large networks, causing delays and poor performance. Automated re-scheduling tools have shown the potential to limit such undesirable outcomes. This study presents the network-wide effects of local deployment of an adaptive traffic controller for real-time operations that is built on approximate dynamic programming (ADP). The controller aims to limit train delays by advantageously controlling the sequencing of trains at critical locations. By using an approximation to the optimised value function of dynamic programming that is updated by reinforcement learning techniques, ADP reduces the computational burden substantially. This framework has been established for isolated local control, so here we investigate the effects of distributed deployment. Our ADP controller is interfaced with a microscopic railway traffic simulator to evaluate its effect on a large and dynamic railway system, which controls critical points independently. The proposed approach achieved a reduction in train delays by comparison with First-Come-First-Served control. We also found the improvements to be greater at terminal stations compared to the vicinity of our control areas.

关键词： Rail transportation Real-time systems Delays dynamic programming Rails Aerospace electronics Tools approximate dynamic programming railway traffic management adaptive control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：