检索结果-内蒙古大学图书馆

Adaptive stochastic lookahead policies for dynamic multi-period purchasing and inventory routing

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2024年第3期318卷 1028-1041页

作者： Cuellar-Usaquen, Daniel Ulmer, Marlin W. Gomez, Camilo Alvarez-Martinez, David Univ Los Andes Cra 1 N 18A-12 Bogota 111711 Colombia Otto von Guericke Univ Univ Pl 2 D-39106 Magdeburg Germany

We explore a problem faced by agri-food e-commerce platforms in purchasing different, perishable products and collecting them from multiple producers and delivering them to a single warehouse, aiming to maintain adequate inventory levels to meet current and future customer demand, while avoiding waste. Customer demand and suppliers' purchase prices and supply volumes are uncertain and revealed on a periodical basis. Every period, purchasing, inventory, and routing decisions are made to satisfy demand and to build inventory for future periods. For effective decisions integrating all three decision components and anticipating future developments, we propose a stochastic lookahead method that, in every period, samples future scenarios for demand, supply volumes, and prices. It then solves a two-stage stochastic program to obtain the decision for the current period. To make this approach computationally tractable, we reduce the routing decision in the two-stage program and use an approximate routing cost instead. Given the reduced decision, we then create the final decision via a conventional routing heuristic. We learn the routing cost approximation adaptively via repeated training simulations. In comprehensive experiments, we show that all three components, stochastic lookahead, routing cost approximation, and adaptive learning, are very effective individually, but especially in combination. We also provide a comprehensive analysis of the problem parameters and obtain valuable insights in problem and methodology.

关键词： Routing Agri-food supply chains dynamic multi-period vehicle routing approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate optimal and safe coordination of nonlinear second-order multirobot systems with model uncertainties

引用

ISA TRANSACTIONS 2024年 149卷 155-167页

作者： Guo, Yaohua Huang, He Northwestern Polytech Univ 127 Youyi Rd Xian 710072 Shaanxi Peoples R China

This paper investigates the approximate optimal coordination for nonlinear uncertain second -order multirobot systems with guaranteed safety (collision avoidance) Through constructing novel local error signals, the collision -free control objective is formulated into an coordination optimization problem for nominal multirobot systems. Based on approximate dynamic programming technique, the optimal value functions and control policies are learned by simplified critic -only neural networks (NNs). Then, the approximated optimal controllers are redesigned using adaptive law to handle the effects of robots' uncertain dynamics. It is shown that the NN weights estimation errors are uniformly ultimately bounded under proper conditions, and safe coordination of multiple robots can be achieved regardless of model uncertainties. Numerical simulations finally illustrate the effectiveness of the proposed controller.

关键词： approximate dynamic programming Collision-free coordination Model uncertainties Multirobot systems

来源：评论

学校读者我要写书评

暂无评论

Math-based reinforcement learning for the adaptive budgeted influence maximization problem

引用

NETWORKS 2024年第3期83卷 547-569页

作者： Fadda, Edoardo Di Corso, Evelina Brusco, Davide Aelenei, Vlad Stefan Rares, Alexandru Balan Politecn Torino Dept Math Sci Turin Italy ISIRES Ist Italiano Ric & Sviluppo Turin Italy Politecn Torino Dept Math Sci Corso Duca Abruzzi 24 I-10129 Turin Italy

In social networks, the influence maximization problem requires selecting an initial set of nodes to influence so that the spread of influence can reach its maximum under certain diffusion models. Usually, the problem is formulated in a two-stage un-budgeted fashion: The decision maker selects a given number of nodes to influence and observes the results. In the adaptive version of the problem, it is possible to select the nodes at each time step of a given time interval. This allows the decision-maker to exploit the observation of the propagation and to make better decisions. This paper considers the adaptive budgeted influence maximization problem, that is, the adaptive problem in which the decision maker has a finite budget to influence the nodes, and each node requires a cost to be influenced. We present two solution techniques: The first is an approximated value iteration leveraging mixed integer linear problems while the second exploits new concepts from graph neural networks. Extensive numerical experiments demonstrate the effectiveness of the proposed approaches.

关键词： adaptive budgeted influence maximization problem approximate dynamic programming graph neural networks mixed integer linear problem reinforcement learning two-stage stochastic problem

来源：评论

学校读者我要写书评

暂无评论

A Q-Learning Algorithm to Solve the Two-Player Zero-Sum Game Problem for Nonlinear Systems

引用

INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING 2025年第3期39卷 566-581页

作者： Islam, Afreen Chen, Anthony Siming Herrmann, Guido Univ Manchester Dept Elect & Elect Engn Manchester England

This paper deals with the two-player zero-sum game problem, which is a bounded L2$$ {L}_2 $$-gain robust control problem. Finding an analytical solution to the complex Hamilton-Jacobi-Issacs (HJI) equation is a challenging task. Hence, a novel Q-learning algorithm for unknown continuous-time (CT) affine-in-inputs nonlinear systems is proposed for generating an approximate solution to the HJI equation, which is valid in a local domain due to the use of a local approximator, that is, a Neural Network (NN) structure. The approach is model-free and does not require the knowledge of system drift dynamics, and input and disturbance gains. The algorithm learns online from measurements of state variables in real time. To generate the local approximate solution of the HJI equation for the two-player zero-sum game problem for nonlinear systems, the proposed non-iterative algorithm requires only a single critic NN instead of the commonly used triple NN approximator structure. A persistence of excitation condition is required to guarantee Uniformly Ultimately Boundedness (UUB) and convergence to the optimal solution. The effectiveness of the proposed Q-learning approach for the two-player zero-sum game problem is demonstrated via simulations of a linear F-16 aircraft plant and a highly complex nonlinear system. Proof of closed-loop system stability is provided using Lyapunov Analysis, and convergence of the approximate solution to the true saddle-point solution is guaranteed in a UUB-sense.

关键词： adaptive control approximate dynamic programming Q-learning zero-sum game

来源：评论

学校读者我要写书评

暂无评论

Scheduling autonomous robots for an intralogistic application: A comparison of lookahead-based ADP strategies

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 271卷

作者： Battistotti, Margherita Brandimarte, Paolo Giancola, Francesca Mazzi, Nicolo Univ Manchester AMBS Booth St West Manchester M15 6PB England Politecn Torino DISMA Corso Duca Abruzzi 24 I-10129 Turin Italy Spindox Spa via Bisceglie 76 I-20152 Milan Italy Spindox Labs Srl via Cascata 56C I-38123 Trento Italy

The increasing role of e-commerce has spurred a significant amount of research on optimization in warehousing management, including routing and scheduling issues. When material handling is rigidly automated, a deterministic scheduling problem arises, for which solution strategies have been proposed in the literature. A recent trend is the introduction of autonomous robots, which may interact with human operators and offer additional flexibility in item manipulation. The resulting problem is affected by uncertainty, due to the interaction between robots and human workers and the possible failure in items manipulation. In the paper, we propose an adaptation of approximate dynamic programming strategies with limited lookahead, namely, rollout strategies and Monte Carlo tree search. The idea can be interpreted as an intermediate approach between the solution of a deterministic problem, disregarding uncertainty and using a long lookahead, or the application of pure state-based dispatching rules with no lookahead. The proposed approaches are compared against exact dynamic programming on small-size instances, and then evaluated on larger instances, proving their viability.

关键词： Intralogistics Scheduling approximate dynamic programming Rollout strategies Monte Carlo tree search

来源：评论

学校读者我要写书评

暂无评论

ADP- and rollout-based dynamic vehicle routing for pick-up service via budgeting capacity

引用

FLEXIBLE SERVICES AND MANUFACTURING JOURNAL 2024年第2期37卷 513-557页

作者： Wu, Yu Zeng, Bo Jian, Ming Southwest Jiaotong Univ Sch Transportat & Logist Chengdu 611756 Sichuan Peoples R China Southwest Jiaotong Univ Natl United Engn Lab Integrated & Intelligent Tran Chengdu 611756 Sichuan Peoples R China Southwest Jiaotong Univ Natl Engn Lab Big Data Applicat Integrated Transpo Chengdu 611756 Sichuan Peoples R China Univ Pittsburgh Dept Ind Engn Pittsburgh PA 15261 USA

We address a dynamic parcel pick-up problem for a capacitated vehicle, which involves collecting customers' parcels and handling service cancellations and requests. In the existing studies, service cancellation and/or the reuse of the released capacity from cancellation have not been considered by state-based decisions (i.e., decision policy) with incorporating stochastic information into decision-making. Aimed at minimizing the expected total travel distance while maximizing vehicle capacity usage, we formulate the problem as a Markov decision process and develop an approximate dynamic programming method. First, we aggregate the post-decision state, consisting of multiple numerical and set components, into a two-dimensional numerical vector by budgeting the vehicle's capacity for future requests. This results in a decrease in the size of the state searching space from exponential to quadratic. Second, we use the aggregation vector to approximate the post-decision state and obtain an offline policy in a lookup-table representation through approximate Value Iteration. Third, we apply the post-decision rollout algorithm online with the offline policy as its base policy, resulting in an online rollout policy. To evaluate our approaches, we conduct two sets of computational experiments with two stepsize rules (i.e., constant stepsize rule and 1/n stepsize rule) together with a epsilon\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon$$\end{document}-heuristic exploration measure adopted in developing offline base policy. The first set benchmarks a dynamic programming-solvable optimal policy on a simplified model to evaluate the optimality degree of our solving method. The second set compares our rollout policy with both the counterpart base policy and a 2-Opt-based heuristic, verifi

关键词： Pick-up service approximate dynamic programming Rollout Capacity budget Stochastic request Service cancellation Service opportunity transfer

来源：评论

学校读者我要写书评

暂无评论

Markdown Optimization via approximate dynamic programming

引用

INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS 2013年第1期6卷 64-78页

作者： Cosgun, Ozlem Kula, Ufuk Kahraman, Cengiz Fatih Univ Dept Ind Engn TR-34500 Istanbul Turkey Sakarya Univ Dept Ind Engn Sakarya Turkey Istanbul Tech Univ Dept Ind Engn TR-80626 Istanbul Turkey

We consider the markdown optimization problem faced by the leading apparel retail chain. Because of substitution among products the markdown policy of one product affects the sales of other products. Therefore, markdown policies for product groups having a significant crossprice elasticity among each other should be jointly determined. Since the state space of the problem is very huge, we use approximate dynamic programming. Finally, we provide insights on the behavior of how each product price affects the markdown policy.

关键词： approximate dynamic programming Multinomial logit model Cross-price elasticity Markdown optimization Pricing

来源：评论

学校读者我要写书评

暂无评论

approximate dynamic programming Algorithms for Multidimensional Inventory Optimization Problems

引用

IFAC Proceedings Volumes 2013年第9期46卷 2015-2020页

作者： Mustafa Cimen Christopher Kirkbride Lancaster University Management School Lancaster LA1 4YX UK

An important issue in the supply chain literature concerns the optimization of inventory decisions. Single-product inventory problems are widely studied and have been optimally solved under a variety of assumptions. However, as supply chain systems become more complex, inventory decisions become more complicated for which the methods/approaches for optimizing single-product inventory systems are incapable of deriving optimal policies. Manufacturing process flexibility provides an example of such complex application areas. Interrelated products and production facilities form a highly multidimensional, non-decomposable system for which optimal policies cannot be obtained by classical methods. We propose the methodology of approximate dynamic programming (ADP) to overcome the computational challenge imposed by this multidimensionality. Incorporating a sample backup approach, ADP develops policies by utilizing only a fraction of the computations required by classical dynamic programming. However, there are no studies in the literature that optimize production decisions in a stochastic, multifactory, multiproduct inventory system of this complexity. This paper aims to explore the feasibility of ADP algorithms for this application. We present the results from a series of numerical experiments that establish the strong performance of policies developed via temporal difference ADP algorithms in comparison to optimal policies.

关键词： approximate dynamic programming dynamic programming flexible manufacturing systems inventory control Markov decision problems process flexibility simulation optimization

来源：评论

学校读者我要写书评

暂无评论

Self-Guided approximate Linear Programs: Randomized Multi-Shot Approximation of Discounted Cost Markov Decision Processes

引用

MANAGEMENT SCIENCE 2025年第4期71卷 iv-vi, 2751-3636页

作者： Pakiman, Parshan Nadarajah, Selvaprabu Soheili, Negar Lin, Qihang Univ Illinois Coll Business Adm Chicago IL 60607 USA Univ Iowa Tippie Coll Business Iowa City IA 52242 USA

approximate linear programs (ALPs) are well-known models based on value function approximations (VFAs) to obtain policies and lower bounds on the optimal policy cost of discounted-cost Markov decision processes (MDPs). Formulating an ALP requires (i) basis functions, the linear combination of which defines the VFA, and (ii) a state relevance distribution, which determines the relative importance of different states in the ALP objective for the purpose of minimizing VFA error. Both of these choices are typically heuristic;basis function selection relies on domain knowledge, whereas the state-relevance distribution is specified using the frequency of states visited by a baseline policy. We propose a self-guided sequence of ALPs that embeds random basis functions obtained via inexpensive sampling and uses the known VFA from the previous iteration to guide VFA computation in the current iteration. In other words, this sequence takes multiple shots randomly approximating the MDP value function with VFA-based guidance between consecutive approximation attempts. Self-guided ALPs mitigate domain knowledge during basis function selection and the impact of the state-relevance-distribution choice, thus reducing the ALP implementation burden. We establish high-probability error bounds on the VFAs from this sequence and show that a worst-case measure of policy performance improved. We find that these favorable implementation and theoretical properties translate to encouraging numerical results on perishable inventory control and options pricing applications, where self-guided ALP policies improve upon policies from problem-specific methods. More broadly, our research takes a meaningful step toward application-agnostic policies and bounds for MDPs.

关键词： approximate linear programming random features Markov decision processes approximate dynamic programming reinforcement learning inventory control options pricing

来源：评论

学校读者我要写书评

暂无评论

Planning Versus Learning: Fair Space-Time Scheduling for Unwired Networks

引用

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS 2024年第11期23卷 16621-16634页

作者： Peng, Chen Mitra, Urbashi Univ Southern Calif Dept Elect & Comp Engn Los Angeles CA 90089 USA

Space-time scheduling for multi-user networks under fairness considerations is investigated. Scheduling is formulated as a sequential decision-making problem under the Markov Decision Processes (MDP) framework. Although the initial focus of the work is underwater acoustic networks, the proposed strategies are also validated for terrestrial radio frequency networks. If environment exploration is expensive, planning is more efficient than online learning. A challenge of the proportional fairness is that the additive structure between current and future rewards does not hold. An approximate reward function that is additive is proposed, enabling dynamic programming. Computational complexity is addressed through sample-based approximations. Error accumulation and error bounds are analyzed to show that error decays with time. As mobility induces model-shifts, a novel re-planning scheme is proposed to optimize the timings of policy updates. Numerical results show that the proposed scheme significantly improves network capacity while maintaining a high level of fairness. Furthermore, the proposed approach yields average capacity and fairness gains as high as 37% and 27%, respectively, compared to current approaches.

关键词： Underwater acoustics Planning Radio frequency Array signal processing Numerical models Multiaccess communication Mathematical models Network scheduling fairness Markov decision process approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：