检索结果-内蒙古大学图书馆

Enhanced Exploration Least-Squares Methods for Optimal Stopping Problems

IEEE CONTROL SYSTEMS LETTERS 2022年 6卷 271-276页

作者： Forootani, Ali Tipaldi, Massimo Iervolino, Raffaele Dey, Subhrakanti Maynooth Univ Hamilton Inst Maynooth W23 F2K8 Kildare Ireland Univ Sannio Dept Engn I-82100 Benevento Italy Univ Naples Federico II Dept Elect Engn & Informat Technol I-80125 Naples Italy

This letter presents an approximate dynamic programming (ADP) least-squares based approach for solving optimal stopping problems with a large state space. By extending some previous work in the area of optimal stopping problems, it provides a framework for their formulation and resolution. The proposed method uses a combined on/off policy exploration mechanism, where states are generated by means of state transition probability distributions different from the ones dictated by the underlying Markov decision processes. The contraction mapping property of the associated projected Bellman operator is analysed as well as the convergence of the resulting algorithm.

关键词： Markov processes Cost function Probability distribution Steady-state Monte Carlo methods Mathematical model Computational modeling Optimal stopping problem Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming with Bezier Curves/Surfaces for Top-percentile Traffic Routing

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2012年第3期218卷 698-707页

作者： Grothey, Andreas Yang, Xinan Univ Edinburgh Coll Sci & Engn Sch Math Edinburgh EH8 9YL Midlothian Scotland

Multi-homing is used by Internet Service Providers (ISPs) to connect to the Internet via different network providers. This study develops a routing strategy under multi-homing in the case where network providers charge ISPs according to top-percentile pricing (i.e. based on the 0th highest volume of traffic shipped). We call this problem the Top-percentile Traffic Routing Problem (TpTRP). Solution approaches based on Stochastic dynamic programming require discretization in state space, which introduces a large number of state variables. This is known as the curse of dimensionality in state space. To overcome this, in previous work we have suggested to use approximate dynamic programming (ADP) to construct value function approximations, which allow us to work in continuous state space. The resulting ADP model provides well performing routing policies for medium sized instances of the TpTRP. In this work we extend the ADP model, by using Bezier Curves/Surfaces to obtain continuous-time approximations of the time-dependent ADP parameters. This modification reduces the number of regression parameters to estimate, and thus accelerates the efficiency of parameter training in the solution of the ADP model, which makes realistically sized TpTRP instances tractable. We argue that our routing strategy is near optimal by giving bounds. (C) 2011 Elsevier B.V. All rights reserved.

关键词： Top-percentile pricing Multi-homing Stochastic Routing policy approximate dynamic programming Bezier Curves/Surfaces

来源：评论

学校读者我要写书评

暂无评论

Robot Scheduling for Mobile-Rack Warehouses: Human-Robot Coordinated Order Picking Systems

引用

PRODUCTION AND OPERATIONS MANAGEMENT 2022年第1期31卷 98-116页

作者： Wang, Zheng Sheu, Jiuh-Biing Teo, Chung-Piaw Xue, Guiqin Dalian Maritime Univ Sch Maritime Econ & Management 1 Linghai Rd Dalian 116026 Peoples R China Natl Taiwan Univ Dept Business Adm 1Sect 4Roosevelt Rd Taipei 10617 Taiwan Natl Univ Singapore NUS Business Sch Singapore 119077 Singapore Natl Univ Singapore Inst Operat Res & Analyt Singapore 119077 Singapore

Intelligent part-to-picker systems are spreading across a broad range of industries as preferred solutions for agile order fulfillment, wherein mobile racks are carried by robots and moved to stations where human pickers can pick items from them. Such systems raise the challenge of designing good work schedules for human pickers;they also give rise to a new class of operational scheduling problems in human-robot coordinated order picking systems. This work studies the problem of finding a suitable robot schedule that takes into account the schedule-induced fluctuation of the working states of human pickers. A proposed model enables mobile racks with various workloads to be assigned to pickers, and schedule the racks that are assigned to every picker to minimize the expected total picking time. The problem is formulated as a stochastic dynamic program model. An approximate dynamic programming (ADP)-based branch-and-price solution approach is used to solve this problem. The developed model is calibrated using data that were collected from a dominant e-commerce company in China. Pickers' working state transitions are modeled based on data obtained from this warehouse. Counter-factual studies demonstrate that the proposed approach can solve a moderately sized problem with 50 racks in under 2 minutes. More importantly, the approach yields high-quality solutions with picking times that are 10% shorter than the solutions that did not consider schedule-induced fluctuations of pickers' working states.

关键词： robot scheduling order picking human– machine coordination circadian rhythm approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Generalized Actor-Critic Learning Optimal Control in Smart Home Energy Management

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2021年第10期17卷 6614-6623页

作者： Wei, Qinglai Liao, Zehua Shi, Guang Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China Macau Univ Sci & Technol Inst Syst Engn Macau 999078 Peoples R China Coordinat Ctr China Natl Comp Network Emergency Response Tech Team Beijing 100029 Peoples R China

This article is concerned with a new generalized actor-critic learning (GACL) optimal control method. It aims at the optimal energy control and management for smart home systems, which is expected to minimize the consumption cost for home users. In the present GACL optimal control method, it is the first time that three iteration processes, which are global iteration, local iteration, and interior iteration, respectively, are established to obtain the optimal energy control law. The main contribution of the developed method is to establish a common iteration structure for both value and policy iterations in adaptive dynamic programming based on a control law sequence in each iteration for periodic time-varying systems, instead of a single control law, and simultaneously accelerates the convergence rate. The monotonicity, convergence, and optimality of the iterative value function for the GACL optimal control method are proven. Finally, numerical results and comparisons are displayed to show the superiority of the developed method.

关键词： Optimal control Process control Smart homes dynamic programming Numerical models Iterative methods Informatics Actor-critic learning adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming energy management optimal control smart grid

来源：评论

学校读者我要写书评

暂无评论

Efficient Multi-Step Lookahead Bayesian Optimization with Local Search Constraints 61

Efficient Multi-Step Lookahead Bayesian Optimization with Lo...

引用

IEEE 61st Conference on Decision and Control (CDC)

作者： Paulson, Joel A. Sorouifar, Farshud Chakrabarty, Ankush Ohio State Univ Dept Chem & Biomol Engn Columbus OH 43210 USA Mitsubishi Elect Res Labs Cambridge MA 02139 USA

ISBN: (纸本)9781665467612

Bayesian optimization (BO) is a data-efficient approach for optimizing expensive-to-evaluate black-box functions that suffer from noisy evaluations. Traditional BO algorithms ignore the relationship between consecutive input values, which is known to lead to significant "jumps" in the search space that cannot be implemented in practice, especially in online experimental systems. For example, in performance-driven control applications, large changes in the chosen setpoint parameters may trigger fail-safe mechanisms or lead to violation of critical safety constraints. In such applications, it is necessary to limit the allowable search space at each BO iteration, which can be done by incorporating local search constraints into the original problem setting. In this paper, we show how this novel BO setting can be cast as a Markov decision process (MDP) for which the optimal policy is characterized by a dynamic programming (DP) problem. This DP problem generally being computationally prohibitive to solve, we adopt approximate DP methods such as rollout with fast policy search to derive an efficient multi-step lookahead BO policy. We also propose a novel base policy needed for the rollout algorithm, which explicitly incorporates the local search restrictions in an intuitive manner. Lastly, we show via numerical experiments that our proposed multi-step lookahead BO policy outperforms existing methods on a benchmark non-convex global optimization problem.

关键词： Optimization machine learning Gaussian process reinforcement learning approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

The Bayesian Prophet: A Low-Regret Framework for Online Decision Making

引用

MANAGEMENT SCIENCE 2021年第3期67卷 1368-1391页

作者： Vera, Alberto Banerjee, Siddhartha Cornell Univ Ithaca NY 14853 USA

We develop a new framework for designing online policies given access to an oracle providing statistical information about an off-line benchmark. Having access to such prediction oracles enables simple and natural Bayesian selection policies and raises the question as to how these policies perform in different settings. Our work makes two important contributions toward this question: First, we develop a general technique we call compensated coupling, which can be used to derive bounds on the expected regret (i.e., additive loss with respect to a benchmark) for any online policy and off- line benchmark. Second, using this technique, we show that a natural greedy policy, which we call the Bayes selector, has constant expected regret (i.e., independent of the number of arrivals and resource levels) for a large class of problems we refer to as "online allocation with finite types," which includes widely studied online packing and online matching problems. Our results generalize and simplify several existing results for online packing and online matching and suggest a promising pathway for obtaining oracle-driven policies for other online decision-making settings.

关键词： stochastic optimization prophet inequalities approximate dynamic programming revenue management online allocation online packing online matching

来源：评论

学校读者我要写书评

暂无评论

Robust-adaptive dynamic programming-based time-delay control of autonomous ships under stochastic disturbances using an actor-critic learning algorithm

引用

JOURNAL OF MARINE SCIENCE AND TECHNOLOGY 2021年第4期26卷 1262-1279页

作者： Esfahani, Hossein Nejatbakhsh Szlapczynski, Rafal Gdansk Univ Technol Fac Ocean Engn & Ship Technol Gdansk Poland

This paper proposes a hybrid robust-adaptive learning-based control scheme based on approximate dynamic programming (ADP) for the tracking control of autonomous ship maneuvering. We adopt a Time-Delay Control (TDC) approach, which is known as a simple, practical, model free and roughly robust strategy, combined with an Actor-Critic approximate dynamic programming (ACADP) algorithm as an adaptive part in the proposed hybrid control algorithm. Based on this integration, Actor-Critic Time-Delay Control (AC-TDC) is proposed. It offers a high-performance robust-adaptive control approach for path following of autonomous ships under deterministic and stochastic disturbances induced by the winds, waves, and ocean currents. Computer simulations have been conducted under two different conditions in terms of the deterministic and stochastic disturbances and all simulation results indicate an acceptable performance in tracking of paths for the proposed control algorithm in comparison with the conventional TDC approach.

关键词： Autonomous ships Robust adaptive Actor-critic learning approximate dynamic programming Time-delay control

来源：评论

学校读者我要写书评

暂无评论

A deep real options policy for sequential service region design and timing

引用

COMPUTERS & OPERATIONS RESEARCH 2024年 161卷

作者： Rath, Srushti Chow, Joseph Y. J. New York Univ Univ Transportat Ctr Dept Civil & Urban Engn C2SMARTTandon Sch Engn Brooklyn NY 11201 USA

There is a need for strategic flexibility in well-timed deployment decisions in the design and timing of mobility service regions, i.e. cast as 'real options'. This problem becomes increasingly challenging considering the multiple interacting real options in such deployments. We propose a scalable machine learning (ML) based real options (RO) framework for multi-period sequential service region design and timing problem for mobility-on-demand (MoD) services, framed as a Markov decision process with non-stationary stochastic variables. A value function approximation policy from the literature uses multi-option least squares Monte Carlo simulation to get a policy value for a set of interdependent deployment decisions as deferral options (i.e., CR policy). The objective is to determine the optimal selection and timing of a set of zones to include in a service region (which to add now, which to defer to a later time to reconsider). However, prior work required explicit enumeration of all possible sequences of deployments (i.e. 5 options translate to 120 sequences). To address the combinatorial complexity arising from sequence enumeration, we propose a new variant 'deep' real options policy using an efficient recurrent neural network (RNN) based ML method (i.e., CR - RNN policy) to sample sequences to forego the need for enumeration, making the network design and timing policy tractable for large scale implementation. Experiments based on multiple service region scenarios in New York City demonstrate the efficacy of the proposed policy in substantially reducing the overall computational cost (i.e., time reduction associated with the RO evaluation of more than 90% of total deployment sequences is achieved), with zero to near-zero gap compared to the benchmark. We validate the model in a case study of sequential service region design for expansion of MoD services in Brooklyn, NYC, under service demand uncertainty. Results show that using the CR-RNN policy in determining op

关键词： Sequential service region design Real options policy Recurrent neural networks approximate dynamic programming Flexible deployment Markov decision process Demand uncertainty

来源：评论

学校读者我要写书评

暂无评论

dynamic service of geographically dispersed time-sensitive demands

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2024年 163卷

作者： de Jong, Niels Aslan, Ayse Bakir, Ilke Univ Groningen Dept Operat NL-9700 AV Groningen Netherlands Edinburgh Napier Univ Sch Engn & Built Environm Edinburgh EH10 5DT Scotland

This paper presents a new framework that models the novel dynamic vehicle dispatch problem with holding costs (DVDPHC), which focuses on serving stochastic demands at geographically dispersed locations in a timely manner. This framework is applicable, among others, to the post-disaster ambulance bus routing problem , where an ambulance bus must pick up (urgent) patients at geographically dispersed locations and bring them to a centrally-located hospital as quickly as possible. Solving the DVDPHC requires a dynamic decision-making rule at each decision moment for which demands to serve at the current location, and where to direct the vehicle next. We propose a heuristic based on approximate dynamic programming combined with a neural network (ADP-NN) for effectively solving the DVDPHC. Numerical experiments demonstrate that our proposed method is fast, scalable and robust. Furthermore, it keeps up with computationally heavy direct lookahead (DLA) benchmarks on 120 large representative instances, achieving on average 12.77% total cost improvement. Numerical analysis also reveals that our proposed method exhibits complex self-learned flexible behavior, such as waiting near locations in anticipation of new demand.

关键词： dynamic vehicle routing Vehicle dispatch problem Collection problem Time-sensitive products approximate dynamic programming Genetic algorithm

来源：评论

学校读者我要写书评

暂无评论

Power Control Optimization for Autonomous Hybrid Electric Vehicles with Flexible Driveline Torque Demand

Power Control Optimization for Autonomous Hybrid Electric Ve...

引用

American Control Conference (ACC)

作者： Kargar, Mohammadali Song, Xingyong

ISBN: (数字)9781665451963

ISBN: (纸本)9781665451963

In the study of powertrain controls optimization for autonomous vehicles, the vehicle's external dynamics and powertrain dynamics are often treated separately, where one of the unique features is usually neglected. This uniqueness, which is referred to as flexible power demand, states that the powertrain control does not need to exactly meet the power requested by the vehicle motion controller at every moment. In this study, a method based on the approximate dynamic programming (ADP) is explored to design the powertrain controller, where the flexibility in power demand is incorporated in the ADP framework. At last, a case study is shown to examine the efficacy of the explored method.

关键词： Autonomous vehicles Hybrid electric vehicles Energy Management approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：