检索结果-内蒙古大学图书馆

An ADP framework for flexibility and cost aggregation: Guarantees and open problems

ELECTRIC POWER SYSTEMS RESEARCH 2024年 234卷

作者： Bandeira, Maisa Beraldo Faulwasser, Timm Engelmann, Alexander TU Dortmund Univ Inst Energy Syst Energy Efficiency & Energy Econ i Dortmund Germany

With the increasing amount of Distributed Energy Resources (DERs), coordination of Distribution Grid Operators (DSOs) and Transmission Grid Operators (TSOs) is of paramount importance. Managing a large number of DERs at the TSO level is, however, challenging. To address this problem, flexibility aggregation is a topic of frequent research activities. Aggregation means to describe the combined flexibility of the DERs at the vertical grid coupling between DSO and TSO. Existing works are often limited with respect to guaranteeing feasibility, with respect to efficient numerical implementation, and in terms of quantification of the cost of DER usage. In the present paper, we investigate aggregation based on approximate dynamic programming (ADP). We propose efficient numerical aggregation schemes using tools from computational geometry thus avoiding the need to solve multiple OPF problems. We rely on different variants of the DistFlow model for radial grids, which are computationally efficient. This allows to model of current and voltage limits and enables the consideration of voltage dependencies in the aggregation. Furthermore, we propose a method for cost aggregation and identify open problems of flexibility aggregation.

关键词： Flexibility aggregation Congestion management Feasible operation region TSO-DSO coordination approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

A tutorial on value function approximation for stochastic and dynamic transportation

引用

4OR-A QUARTERLY JOURNAL OF OPERATIONS RESEARCH 2024年第1期22卷 145-173页

作者： Heinold, Arne Univ Kiel Sch Econ & Business Kiel Germany

This paper provides an introductory tutorial on Value Function Approximation (VFA), a solution class from approximate dynamic programming. VFA describes a heuristic way for solving sequential decision processes like a Markov Decision Process. Real-world problems in supply chain management (and beyond) containing dynamic and stochastic elements might be modeled as such processes, but large-scale instances are intractable to be solved to optimality by enumeration due to the curses of dimensionality. VFA can be a proper method for these cases and this tutorial is designed to ease its use in research, practice, and education. For this, the tutorial describes VFA in the context of stochastic and dynamic transportation and makes three main contributions. First, it gives a concise theoretical overview of VFA's fundamental concepts, outlines a generic VFA algorithm, and briefly discusses advanced topics of VFA. Second, the VFA algorithm is applied to the taxicab problem that describes an easy-to-understand transportation planning task. Detailed step-by-step results are presented for a small-scale instance, allowing readers to gain an intuition about VFA's main principles. Third, larger instances are solved by enhancing the basic VFA algorithm demonstrating its general capability to approach more complex problems. The experiments are done with artificial instances and the respective Python scripts are part of an electronic appendix. Overall, the tutorial provides the necessary knowledge to apply VFA to a wide range of stochastic and dynamic settings and addresses likewise researchers, lecturers, tutors, students, and practitioners.

关键词： Tutorial Markov decision process approximate dynamic programming Value function approximation Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

The dynamic Freight Routing Problem for Less-Than-Truckload Carriers

引用

TRANSPORTATION SCIENCE 2023年第3期57卷 717-740页

作者： Baubaid, Ahmad Boland, Natashia Savelsbergh, Martin King Fahd Univ Petr & Minerals Ind & Syst Engn Dept Dhahran 31261 Saudi Arabia King Fahd Univ Petr & Minerals Interdisciplinary Res Ctr Smart Mobil & Logist Dhahran 31261 Saudi Arabia Georgia Inst Technol H Milton Stewart Sch Ind & Syst Engn Atlanta GA 30332 USA

Less-than-truckload (LTL) carriers transport freight shipments from origins to destinations by consolidating freight using a network of terminals. As daily freight quantities are uncertain, carriers dynamically decide freight routes on the day of operations. We introduce the dynamic freight routing problem (DFRP) and model this problem as a Markov decision process (MDP). To overcome the curses of dimensionality of the MDP model, we introduce an approximate dynamic programming (ADP) solution approach that uses a lookup table to store value function approximations and present and compares a number of aggregation approaches that use features of the postdecision states (PDSs) to aggregate the PDS space and reduce the number of entries in the lookup table. Furthermore, because the decision subproblems are integer programs (IPs), we present a framework for integrating lookup tables into the decision subproblem IPs. This framework consists of (1) a modeling approach for the integration of lookup table value function approximations into subproblem IPs to form extended subproblem IPs;(2) a solution approach, PDS-IP-bounding, which decomposes the extended subproblem IPs into many smaller IPs and uses dynamic bounds to reduce the number of small IPs that have to be solved;and (3) an adaptation of the e-greedy exploration-exploitation algorithm for the IP setting. Our computational experiments show that despite the DFRP having high-dimensional PDS vectors, a two-dimensional aggregation of the space can produce policies that outperform standard myopic policies. Moreover, they demonstrate that the PDS-IP-bounding algorithm provides computational advantages over solving the extended subproblem IPs using a commercial solver.

关键词： less-than-truckload freight transportation approximate dynamic programming value function approximation load planning

来源：评论

学校读者我要写书评

暂无评论

Fourier-Hermite dynamic programming for Optimal Control

引用

IEEE TRANSACTIONS ON AUTOMATIC CONTROL 2023年第10期68卷 6377-6384页

作者： Hassan, Syeda Sakira Sarkka, Simo Aalto Univ Dept Elect Engn & Automat Espoo 02150 Finland

In this article, we propose a novel computational method for solving nonlinear optimal control problems. The method is based on the use of Fourier-Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor-series expansion used in differential dynamic programming. The coefficients of the Fourier-Hermite series can be numerically computed by using sigma-point methods, which leads to a novel class of sigma-point-based dynamic programming methods. We also prove the quadratic convergence of the method and experimentally test its performance against other methods.

关键词： approximate dynamic programming differential dynamic programming Fourier-Hermite series sigma-point dynamic programming trajectory optimization

来源：评论

学校读者我要写书评

暂无评论

Self-Guided approximate Linear Programs: Randomized Multi-Shot Approximation of Discounted Cost Markov Decision Processes

引用

MANAGEMENT SCIENCE 2025年第4期71卷 iv-vi, 2751-3636页

作者： Pakiman, Parshan Nadarajah, Selvaprabu Soheili, Negar Lin, Qihang Univ Illinois Coll Business Adm Chicago IL 60607 USA Univ Iowa Tippie Coll Business Iowa City IA 52242 USA

approximate linear programs (ALPs) are well-known models based on value function approximations (VFAs) to obtain policies and lower bounds on the optimal policy cost of discounted-cost Markov decision processes (MDPs). Formulating an ALP requires (i) basis functions, the linear combination of which defines the VFA, and (ii) a state relevance distribution, which determines the relative importance of different states in the ALP objective for the purpose of minimizing VFA error. Both of these choices are typically heuristic;basis function selection relies on domain knowledge, whereas the state-relevance distribution is specified using the frequency of states visited by a baseline policy. We propose a self-guided sequence of ALPs that embeds random basis functions obtained via inexpensive sampling and uses the known VFA from the previous iteration to guide VFA computation in the current iteration. In other words, this sequence takes multiple shots randomly approximating the MDP value function with VFA-based guidance between consecutive approximation attempts. Self-guided ALPs mitigate domain knowledge during basis function selection and the impact of the state-relevance-distribution choice, thus reducing the ALP implementation burden. We establish high-probability error bounds on the VFAs from this sequence and show that a worst-case measure of policy performance improved. We find that these favorable implementation and theoretical properties translate to encouraging numerical results on perishable inventory control and options pricing applications, where self-guided ALP policies improve upon policies from problem-specific methods. More broadly, our research takes a meaningful step toward application-agnostic policies and bounds for MDPs.

关键词： approximate linear programming random features Markov decision processes approximate dynamic programming reinforcement learning inventory control options pricing

来源：评论

学校读者我要写书评

暂无评论

Combined Use of dynamic Inversion and Reinforcement Learning for Motion Control of an Supersonic Transport Aircraft

引用

OPTICAL MEMORY AND NEURAL NETWORKS 2024年第SUPPL3期33卷 S399-S413页

作者： Dhiman, Gaurav Tiumentsev, Yu. V. Tskhai, R. A. Natl Res Univ Moscow Aviat Inst Moscow 125080 Russia

The task of aircraft motion control has to be solved under conditions of numerous heterogeneous uncertainties both in the aircraft motion model and in the environment in which the aircraft is flying. These uncertainties, in particular, are caused by the fact that in the flight of the aircraft can occur various kinds of abnormal situations caused by failures of equipment and systems of the aircraft, damage to the airframe and propulsion system of the aircraft. Some of these failures and damages have a direct impact on the dynamic characteristics of the aircraft as a control object. In this regard, the problem arises of such an adjustment of aircraft control algorithms that would provide the ability to adapt to the changed dynamics of the aircraft. It is extremely difficult, and in some cases impossible, to foresee in advance all possible damages, failures and their combinations. Hence, it is necessary to implement adaptive flight control algorithms that are able to adjust to the changing situation. One of the effective tools for solving such problems is reinforcement learning in the approximate dynamic programming (ADP) variant, in combination with artificial neural networks. In the last decade, a family of methods known as Adaptive Critic Design (ACD) has been actively developed within the ADP approach to control the behavior of complex dynamic systems. In our paper we consider the application of one of the variants of the ACD approach, namely SNAC (Single Network Adaptive Critic) and its development through its joint use with the method of dynamic inversion. The effectiveness of this approach is demonstrated on the example of longitudinal motion control of a supersonic transport airplane.

关键词： aircraft motion control machine learning dynamic inversion approximate dynamic programming adaptive critic design SNAC approach adaptive control

来源：评论

学校读者我要写书评

暂无评论

Robust Near-Optimal Coordination in Uncertain Multiagent Networks With Motion Constraints

引用

IEEE TRANSACTIONS ON CYBERNETICS 2023年第5期53卷 2841-2851页

作者： Guo, Yaohua Chen, Gang Xi An Jiao Tong Univ Sch Aerosp Engn Xian 710049 Peoples R China

This article addresses the robust coordination problem for nonlinear uncertain second-order multiagent networks with motion constraints, including velocity saturation and collision avoidance. A single-critic neural network-based approximate dynamic programming approach and exact estimation of unknown dynamics are employed to learn online the optimal value function and controller. By incorporating avoidance penalties into tracking variable, constructing a novel value function, and designing of suitable learning algorithms, multiagent coordination and collision avoidance are achieved simultaneously. We show that the developed feedback-based coordination strategy guarantees uniformly ultimately bounded convergence of the closed-loop dynamical stability and all underlying motion constraints are always strictly obeyed. The effectiveness of the proposed collision-free coordination law is finally illustrated using numerical simulations.

关键词： Collision avoidance Artificial neural networks Multi-agent systems Heuristic algorithms dynamic programming Uncertainty Robustness approximate dynamic programming collision avoidance near-optimal control robust coordination

来源：评论

学校读者我要写书评

暂无评论

Reductions of non-separable approximate linear programs for network revenue management

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2023年第1期309卷 252-270页

作者： Laumer, Simon Barz, Christiane Univ Zurich Dept Business Adm CH-8032 Zurich Switzerland

We suggest a novel choice of non-separable basis functions for an approximate linear programming approach to the well-known network revenue management problem. Considering non-separability is particularly important when interdependencies between resources are large. Such a situation can be illustrated for example by a bus line, where different origin-destination pairs have many overlapping segments. Traditional separable approximation approaches tend to ignore the resulting interactions. We suggest to group resources into non-separable subnetworks. For each chosen subnetwork, basis functions either span the whole function space or consist of linear functions. Given this more general choice of basis functions, we extend existing reductions of approximate linear programs. If there is only one subnetwork, for which the basis functions span the whole function space, we prove the equivalence to a compact linear program of polynomial size. For the general case, we suggest an approximate reduction. Numerical examples illustrate our novel upper bounds for the maximum expected revenue and the corresponding competitive policies. In particular, we find that the added benefit of non-separability heavily depends on the network structure and the capacity. Our work helps to better understand the impact of assuming separability in network revenue management. The polynomial sized reductions make it possible to estimate the added average revenue resulting from incorporating interactions between resources. The theory we develop demonstrates how the interpretation of dual variables as state-action probabilities can be applied to reduce exponentially large approximate linear programs via variable aggregation.

关键词： Revenue management approximate dynamic programming Network revenue management Reductions Non-separability

来源：评论

学校读者我要写书评

暂无评论

Optimizing last-mile delivery: a dynamic compensation strategy for occasional drivers

引用

OR SPECTRUM 2024年 1-58页

作者： Schur, Rouven Winheller, Kai Univ Duisburg Essen Chair Prod & Logist Planning Mercator Sch Management Lotharstr 65 D-47057 Duisburg Germany

Amid the rapid growth of online retail, last-mile delivery faces significant challenges, including the cost-effective delivery of goods to all delivery locations. Our work contributes to this stream by applying dynamic pricing techniques to effectively model the possible involvement of the crowd in fulfilling delivery tasks. The use of occasional drivers (ODs) as a viable, cost-effective alternative to traditional dedicated drivers (DDs) prompts the necessity to focus on the inherent challenge posed by the uncertainty of ODs' arrival times and willingness to perform deliveries. We introduce a dynamic programming framework that offers individualized bundles of a delivery task and compensation to ODs as they arrive. This model, akin to a reversed form of dynamic pricing, accounts for ODs' decision-making by treating their acceptance thresholds as a random variable. Therefore, our model addresses the dynamic and stochastic nature of OD availability and decision-making. We analytically solve the stage-wise optimization problem, outline inherent challenges such as the curses of dimensionality, and present structural properties. Tailored to meet these challenges, our approximation methods aim to accurately determine avoided costs, which are a key factor in calculating optimal compensation. Our simulation study reveals that the savings generated by involving ODs in deliveries can be significantly increased through our individualized dynamic compensation policy. This approach not only excels in generating savings for the firm but also provides a utility surplus for ODs. Additionally, we demonstrate the applicability of our approach to scenarios with time windows and illustrate the trade-off that arises from time window partitioning.

关键词： dynamic pricing Last-mile delivery Occasional drivers approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

A Bayesian learning and pricing model with multiple unknown demand parameters

引用

ANNALS OF OPERATIONS RESEARCH 2024年第1期343卷 493-513页

作者： Xiao, Baichun Yang, Wei Long Isl Univ Coll Management CW Post Brookville NY 11548 USA

This article presents a Bayesian learning model for demand estimation in revenue management. Different from most existing models in the literature, our discussion centers on demand functions with an arbitrary number of unknown and correlated parameters, and estimating them simultaneously. We formulate the problem as a Dirichlet learning model and show the search process converges to the true parameter values. As the observed data does not unambiguously reveal the underlying demand curve, the exploration scheme is notably different from conventional Dirichlet sampling process. We apply a partially observable Markov decision process to ensure the true demand curve surfaces as a favorite. Our pricing policy during the learning phase also differs from myopic heuristics by taking both the remaining time and unsold items into consideration. As incomplete learning remains a concern for all existing learning models, we show that the occurrence of uninformative prices is rooted in the dynamics of pricing, and prove that the proposed model is immune from incomplete learning. For revenue performance, the regret bounds established are comparable to the benchmark in the literature under similar conditions. Overall, the proposed model integrates the learning process with earning goals and offers a promising tool to achieve both targets.

关键词： Bayesian demand estimation Pricing approximate dynamic programming Incomplete learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：