检索结果-内蒙古大学图书馆

Data-driven optimal control with a relaxed linear program

AUTOMATICA 2022年 136卷 110052-110052页

作者： Martinelli, Andrea Gargiani, Matilde Lygeros, John Swiss Fed Inst Technol Automatic Control Lab Physikstrasse 3 CH-8092 Zurich Switzerland

The linear programming (LP) approach has a long history in the theory of approximate dynamic programming. When it comes to computation, however, the LP approach often suffers from poor scalability. In this work, we introduce a relaxed version of the Bellman operator for q-functions and prove that it is still a monotone contraction mapping with a unique fixed point. In the spirit of the LP approach, we exploit the new operator to build a relaxed linear program (RLP). Compared to the standard LP formulation, our RLP has only one family of constraints and half the decision variables, making it more scalable and computationally efficient. For deterministic systems, the RLP trivially returns the correct q-function. For stochastic linear systems in continuous spaces, the solution to the RLP preserves the minimizer of the optimal q-function, hence retrieves the optimal policy. Theoretical results are backed up in simulation where we solve sampled versions of the LPs with data collected by interacting with the environment. For general nonlinear systems, we observe that the RLP again tends to preserve the minimizers of the solution to the LP, though the relative performance is influenced by the specific geometry of the problem. (c) 2021 The Authors. Published by Elsevier Ltd.

关键词： approximate dynamic programming Optimal control Data-driven control Linear programming Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

Stochastic one-way carsharing systems with dynamic relocation incentives through preference learning

引用

TRANSPORTATION RESEARCH PART E-LOGISTICS AND TRANSPORTATION REVIEW 2022年第0期166卷

作者： Liu, Yang Xie, Jiaohong Chen, Nan Natl Univ Singapore Dept Ind Syst Engn & Management 1 Engn Dr 2 Singapore 117576 Singapore Natl Univ Singapore Dept Civil & Environm Engn 1 Engn Dr 2 Singapore 117576 Singapore

The development of carsharing services is expected to achieve greater resource efficiency and provide a sustainable solution for future mobility systems. However, operators inevitably face the imbalance between demand and supply in one-way carsharing systems (CSSs). Also, it is challenging for them to make quick and efficient operational decisions when both travel time and trip requests are uncertain. This study leverages historical and online data and proposes a learning-based methodology to quickly make real-time decisions for CSSs, including vehicle assignment, relocation, and user incentive decisions. Compared to the literature in approximate dynamic programming (ADP), which mostly focuses on uncertainty in trip requests, we propose an offline-online ADP approach to consider the temporal and spatial uncertainty in both trip requests and travel time. To tackle our high-dimensional problem, we integrate an online look-ahead policy into the offline value function approximation (VFA) policy to produce a computational tractable high-quality dynamic fleet management policy. Furthermore, a user -based relocation strategy is investigated to rebalance the fleet distribution to meet the demand better. Specifically, we aim to solve the optimal incentives the operator could offer to users to relocate cars, while user preferences towards relocation incentives are generally unknown in practice. We further enhance our anticipatory policy by developing an online module via Bayesian learning that learns the preference model on the fly using users' revealed preference data collected online. The numerical experiments in Singapore demonstrate our offline-online ADP approach improves the solution quality and significantly compared to offline VFA policy. The results also confirm the importance of incorporating uncertainty in travel time. The benefits of the user-based relocation scheme using the Bayesian learning method using online data to enhance anticipatory decisions and learn un

关键词： One-way carsharing approximate dynamic programming Online decision-making User-based vehicle relocation User preference learning Urban mobility operations

来源：评论

学校读者我要写书评

暂无评论

An expected-cost realization-probability optimization approach for the dynamic energy management of microgrid

引用

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS 2022年 136卷 107620-107620页

作者： Zhu, Jianquan Zhuo, Yelin Chen, Jiajun Guo, Ye Mo, Xiemin Liu, Mingbo South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China

dynamic energy management (DEM) is being recognized as an important tool for the economical and efficient operation of microgrid. This paper proposes an expected-cost realization-probability (ECRP) optimization approach for the DEM of microgrid. In the model, the expected cost is taken as an economic goal to find the schedule with the maximum ECRP (i.e., the maximum probability of obtaining an actual operation cost which is lower than this economic goal). It is a stochastic mixed-integer nonlinear programming (MINLP) problem composed of the AC power flow, battery operation characters and logical variables. To solve this NP-hard problem, an ECRP-based approximate dynamic programming is proposed. The original MINLP problem is decomposed into small nonlinear programming subproblems to facilitate the solution. And the ECRP-based value function is formed to describe the relationship among the decisions of subproblems, which helps to obtain the near-optimal decision in a decomposed manner. Simulation examples presented on a real microgrid indicate that the ECRP of the proposed method is higher than those of the other two comparison algorithms.

关键词： approximate dynamic programming dynamic energy management Expected cost Microgrid Realization probability

来源：评论

学校读者我要写书评

暂无评论

dynamic stochastic electric vehicle routing with safe reinforcement learning

引用

TRANSPORTATION RESEARCH PART E-LOGISTICS AND TRANSPORTATION REVIEW 2022年 157卷 102496-102496页

作者： Basso, Rafael Kulcsar, Balazs Sanchez-Diaz, Ivan Qu, Xiaobo Volvo Grp Trucks Technol Gothenburg Sweden Chalmers Univ Technol Elect Engn Gothenburg Sweden Chalmers Univ Technol Technol Management & Econ Gothenburg Sweden Chalmers Univ Technol Architecture & Civil Engn Gothenburg Sweden

dynamic routing of electric commercial vehicles can be a challenging problem since besidesthe uncertainty of energy consumption there are also random customer requests. This paperintroduces the dynamic Stochastic Electric Vehicle Routing Problem (DS-EVRP). A Safe Rein-forcement Learning method is proposed for solving the problem. The objective is to minimizeexpected energy consumption in a safe way, which means also minimizing the risk of batterydepletion while en route by planning charging whenever necessary. The key idea is to learnoffline about the stochastic customer requests and energy consumption using Monte Carlosimulations, to be able to plan the route predictively and safely online. The method is evaluatedusing simulations based on energy consumption data from a realistic traffic model for the cityof Luxembourg and a high-fidelity vehicle model. The results indicate that it is possible tosave energy at the same time maintaining reliability by planning the routes and charging in ananticipative way. The proposed method has the potential to improve transport operations withelectric commercial vehicles capitalizing on their environmental benefits.

关键词： Reinforcement learning approximate dynamic programming Electric vehicles Energy consumption Vehicle routing Green logistics

来源：评论

学校读者我要写书评

暂无评论

On the Taylor Expansion of Value Functions

引用

OPERATIONS RESEARCH 2020年第2期68卷 631-654页

作者： Braverman, Anton Gurvich, Itai Huang, Junfei Northwestern Univ Kellogg Sch Management Evanston IL 60208 USA Cornell Sch Operat Res & Informat Engn New York NY 10044 USA Cornell Tech New York NY 10044 USA Chinese Univ Hong Kong CUHK Business Sch Dept Decis Sci & Managerial Econ Shatin Hong Kong Peoples R China

We introduce a framework for approximate dynamic programming that we apply to discrete-time chains on Z(+)(d) with countable action sets. The framework is grounded in the approximation of the (controlled) chain's generator by that of another Markov process. In simple terms, our approach stipulates applying a second-order Taylor expansion to the value function, replacing the Bellman equation with one in continuous space and time in which the transition matrix is reduced to its first and second moments. In some cases, the resulting equation can be interpreted as a Hamilton-Jacobi-Bellman equation for a Brownian control problem. When tractable, the "Taylored" equation serves as a useful modeling tool. More generally, it is a starting point for approximation algorithms. We develop bounds on the optimality gap-the suboptimality introduced by using the control produced by the Taylored equation. These bounds can be viewed as a conceptual underpinning, analytical rather than relying on weak convergence arguments, for the good performance of controls derived from Brownian approximations. We prove that under suitable conditions and for suitably "large" initial states, (1) the optimality gap is smaller than a 1-alpha fraction of the optimal value, with which alpha is an element of (0, 1) is the discount factor, and (2) the gap can be further expressed as the infinite-horizon discounted value with a "lowerorder" per-period reward. Computationally, our framework leads to an "aggregation" approach with performance guarantees. Although the guarantees are grounded in partial differential equation theory, the practical use of this approach requires no knowledge of that theory.

关键词： Taylor expansion Markov decision process Hamilton-Jacobi-Bellman equation approximate dynamic programming approximate policy iteration

来源：评论

学校读者我要写书评

暂无评论

dynamic Fleet Management for Autonomous Vehicles: Learning- and optimization-based strategies

Dynamic Fleet Management for Autonomous Vehicles: Learning- ...

引用

作者： Alves Beirigo, B. Delft University of Technology

学位级别：博士

Autonomous vehicles (AVs) have been heralded as the key to unlock a shared mobility future where transportation is more efficient, convenient, and cheaper. However, the AV utopia can only come to fruition if the majority of users trust that autonomous mobility-on-demand (AMoD) systems are on a par with owning a vehicle in terms of service quality. Once the perception of quality is highly subjective, we propose a more personalized approach to on-demand mobility, in which users are segmented into service quality classes. These classes comprise minimum requirements regarding responsiveness and privacy, allowing us to model a series of user profiles formalized using strict service quality contracts. By honoring these contracts, providers can build users' trust and gain their loyalty, which on a grander scheme can contribute to a faster transition to a shared mobility *** thesis presents a series of strategies to guaranteeing service quality throughout operational scenarios arising in the timeline of AV technology deployment. First, a precondition to providing service quality in autonomous transportation is safety. During a transition phase to full automation, AV operation will likely be restricted to areas where safe operations are guaranteed, leading to the formation of hybrid street networks comprised of autonomous and non-autonomous vehicle zones. In this setting, meeting user service quality expectations is primarily a matter of coverage, once mobility services will have to access both AV-ready and not AV-ready areas. Accordingly, this thesis proposes solutions to overcome the challenges entailed by such a transition scenario, where infrastructures, regulatory measures, and AV technology are gradually ***, assuming that widespread automated driving is the new status quo, we set out to model rich autonomous transportation scenarios comprised of heterogeneous users and vehicles. Central to our analysis is finding an adequate tradeoff between fleet si

关键词： dynamic fleet management autonomous vehicles service quality on-demand hiring autonomous vehicle zone people and freight integration mobility poverty mobility-on-demand approximate dynamic programming matheuristic mixed integer programming

来源：评论

学校读者我要写书评

暂无评论

Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2012年第3期219卷 611-621页

作者： Schmid, Verena Univ Vienna Dept Business Adm A-1210 Vienna Austria

Emergency service providers are supposed to locate ambulances such that in case of emergency patients can be reached in a time-efficient manner. Two fundamental decisions and choices need to be made real-time. First of all immediately after a request emerges an appropriate vehicle needs to be dispatched and send to the requests' site. After having served a request the vehicle needs to be relocated to its next waiting location. We are going to propose a model and solve the underlying optimization problem using approximate dynamic programming (ADP), an emerging and powerful tool for solving stochastic and dynamic problems typically arising in the field of operations research. Empirical tests based on real data from the city of Vienna indicate that by deviating from the classical dispatching rules the average response time can be decreased from 4.60 to 4.01 minutes, which corresponds to an improvement of 12.89%. Furthermore we are going to show that it is essential to consider time-dependent information such as travel times and changes with respect to the request volume explicitly. Ignoring the current time and its consequences thereafter during the stage of modeling and optimization leads to suboptimal decisions. (C) 2011 Elsevier B.V. All rights reserved.

关键词： OR in health services Emergency vehicles Ambulance location approximate dynamic programming Stochastic optimization

来源：评论

学校读者我要写书评

暂无评论

State of the Art of Adaptive dynamic programming and Reinforcement Learning

引用

CAAI Artificial Intelligence Research 2022年第2期1卷 93-110页

作者： Derong Liu Mingming Ha Shan Xue Department of Mechanical and Energy Engineering Southern University of Science and TechnologyShenzhen 518055China Department of Electrical and Computer Engineering University of Illinois at ChicagoIL 606071USA School of Automation and Electrical Engineering University of Science and Technology BeijingBeijing 100083China School of Computer Science and Engineering South China University of TechnologyGuangzhou 510006China

This article introduces the state-of-the-art development of adaptive dynamic programming and reinforcement learning(ADPRL).First,algorithms in reinforcement learning(RL)are introduced and their roots in dynamic programming are *** dynamic programming(ADP)is then introduced following a brief discussion of dynamic *** in ADP and RL have enjoyed the fast developments of the past decade from algorithms,to convergence and optimality analyses,and to stability *** key steps in the recent theoretical developments of ADPRL are mentioned with some future *** particular,convergence and optimality results of value iteration and policy iteration are reviewed,followed by an introduction to the most recent results on stability analysis of value iteration algorithms.

关键词： adaptive dynamic programming approximate dynamic programming adaptive critic designs neuro-dynamic programming neural dynamic programming reinforcement learning intelligent control learning control optimal control

来源：评论

学校读者我要写书评

暂无评论

A reinforcement learning approach to the stochastic cutting stock problem

引用

EURO JOURNAL ON COMPUTATIONAL OPTIMIZATION 2022年 10卷

作者： Pitombeira-Neto, Anselmo R. Murta, Arthur H. F. Univ Fed Ceara Dept Ind Engn Campus Pici Fortaleza Brazil Univ Fed Ceara Grad Program Modeling & Quantitat Methods Fortaleza Brazil Univ Fed Ceara OPL Lab Operat Res Prod & Logist Fortaleza Brazil

We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock in anticipation of the unknown demand. An optimal solution corresponds to a policy that associates each state with a decision and minimizes the expected total cost. Since exact algorithms scale exponentially with the state-space dimension, we develop a heuristic solution approach based on reinforcement learning. We propose an approximate policy iteration algorithm in which we apply a linear model to approximate the action-value function of a policy. Policy evaluation is performed by solving the projected Bellman equation from a sample of state transitions, decisions and costs obtained by simulation. Due to the large decision space, policy improvement is performed via the cross-entropy method. Computational experiments are carried out with the use of realistic data to illustrate the application of the algorithm. Heuristic policies obtained with polynomial and Fourier basis functions are compared with myopic and random policies. Results indicate the possibility of obtaining policies capable of adequately controlling inventories with an average cost up to 80% lower than the cost obtained by a myopic policy.(c) 2022 The Author(s). Published by Elsevier Ltd on behalf of Association of European Operational Research Societies (EURO).

关键词： Cutting stock problem Reinforcement learning approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Differentiated Pricing of Shared Mobility Systems Considering Network Effects

引用

TRANSPORTATION SCIENCE 2022年第5期56卷 1279页

作者： Soppert, Matthias Steinhardt, Claudius Mueller, Christian Goensch, Jochen Univ Bundeswehr Munich Chair Business Analyt & Management Sci D-85577 Neubiberg Germany Univ Duisburg Essen Chair Serv Operat D-47057 Duisburg Germany

Over the last decades, shared mobility systems have become an integral part of inner-city mobility. Modern systems allow one-way rentals, that is, customers can drop off the vehicle at a different location to where they began their trip. A prominent example is car sharing. Indeed, this work was motivated by the insight we gained in collaborating closely with Europe's largest car sharing provider, Share Now. In car sharing, as well as in shared mobility systems in general, pricing optimization has turned out to be a promising means of increasing profit while challenged by limited vehicle supply and asymmetric demand across time and space. Thus, in practice, providers increasingly use minute pricing that is differentiated according to where a rental originates, that is, considering its location and the time of day. In research, however, such approaches have not been considered yet. In this paper, we therefore introduce the corresponding origin-based differentiated, profit-maximizing pricing problem for shared mobility systems. The problem is to determine spatially and temporally differentiated minute prices, taking network effects on the supply side and several practice relevant aspects into account. Based on a deterministic network flow model, we formulate the problem as a mixed-integer linear program and prove that it is NP-hard. For its solution, we propose a temporal decomposition approach based on approximate dynamic programming. The approach integrates a value function approximation to incorporate future profits and account for network effects. Extensive computational experiments demonstrate the benefits of capturing such effects in pricing generally, as well as showing our value function approximation's ability to anticipate them precisely. Furthermore, in a case study based on Share Now data from Florence in Italy, we observe profit increases of around 9% compared with constant uniform minute prices, which are still the de facto industry standard.

关键词： shared mobility systems car sharing differentiated pricing origin-based pricing supply-side spatio-temporal network effects approximate dynamic programming optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：