检索结果-内蒙古大学图书馆

Integrating simulation, optimization and reinforcement learning for a general class of stochastic scheduling problems

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 279卷

作者： Xie, Fang Li, Yutong Li, Haitao Yantai Univ Sch Econ & Management Yantai 264005 Shandong Peoples R China Univ Missouri St Louis Supply Chain & Analyt Dept St Louis MO 63121 USA

We study a general class of stochastic scheduling problems that include various machine scheduling and resource-constrained project scheduling problems under uncertainty as special cases. A Markov decision process (MDP) model built upon discrete-time Markov chain is developed for the addressed class of problems. To tackle the curse-of-dimensionalities of obtaining the exact closed-loop policy to the MDP model, we present a general approximate dynamic programming framework that integrates simulation, optimization, and reinforcement learning, called Sim-Opt-RL, to provide quality and computationally tractable closed-loop policy. We implement the Sim-Opt-RL framework for a well-known and well-studied stochastic resource-constrained project scheduling (SRCPSP), with a custom designed genetic algorithm, which outperforms the existing closed-loop algorithm, and is competitive with the state-of-the-art open-loop algorithms for the SRCPSP. It is able to provide quality closedloop solutions for instances with size of up to 120 activities in less than 30 min on average.

关键词： Stochastic scheduling Uncertain activity duration approximate dynamic programming Stochastic resource-constrained project sched- uling Simulation

来源：评论

学校读者我要写书评

暂无评论

Optimizing electrolyzer investments for green hydrogen production under market and technology uncertainties

引用

RENEWABLE ENERGY 2025年 249卷

作者： Trivella, Alessio Univ Twente Ind Engn & Business Informat Syst Hallenweg 17 NL-7522 NH Enschede Netherlands

Motivated by increasing curtailment of renewable energy, negative electricity prices, and growing importance of green hydrogen, many renewable energy producers are considering investing in electrolyzers to produce and sell green hydrogen. However, electrolyzers are still expensive technologies and their profitability is affected by uncertainties in electricity and hydrogen prices, investment cost, and curtailment levels, all of which fluctuate over time. In this paper, we study optimal electrolyzer investments as a real option problem where a renewable energy producer can decide over a planning horizon whether investing in an electrolyzer, its capacity, and whether to couple it with a storage system. Since the resulting dynamic stochastic optimization problem is intractable to solve to optimality, we leverage state-of-the-art approximate dynamic programming (ADP) methods to compute near-optimal investment policies and dual bounds. A computational study is conducted, offering insights into the investment option value under traditional and new ADP policies, and optimal investment decisions in relation to the evolution of the uncertainties, including the expected time to invest, asset size, and optimal electrolyzer utilization. The findings can be valuable for renewable energy producers interested in green hydrogen but lacking tools and insights to assess investment decisions in a complex and evolving energy landscape.

关键词： Real options Renewable energy Electrolyzer Green hydrogen Optimal investment approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Leveraging Time-Causal State Variable Aggregation for Real-Time Schedule of Massive Air Conditioners

引用

IEEE TRANSACTIONS ON SMART GRID 2025年第3期16卷 2389-2403页

作者： Liu, Jingguan Ai, Xiaomeng Cui, Shichang Xue, Xizhen Wang, Shengshi Fang, Jiakun Wen, Jinyu Shi, Yang Huazhong Univ Sci & Technol State Key Lab Adv Electromagnet Technol Wuhan 430074 Peoples R China Nanyang Technol Univ Sch Elect & Elect Engn Singapore 639798 Singapore Univ Victoria Dept Mech Engn Victoria BC V8W 3P1 Canada

Air conditioner (AC) loads offer promising flexibility for active distribution networks to manage uncertainties, such as those in renewable energy generation, electricity prices, and load demand. However, real-time scheduling of ACs is challenging due to their massive temporal coupling constraints and time-causal uncertainties. To address this, a novel time-causal aggregation-based approximate dynamic programming (TCA-ADP) algorithm is proposed for efficient scheduling. The time-causality requirements for aggregating state variables are first analyzed to align with the real-time sequential decision-making process. Subsequently, an enhanced aggregation model is developed to ensure both high accuracy and adherence to time causality. The aggregation process is further reformulated as a linear program to optimize aggregation parameters and enable tractable computation. Accordingly, the TCA-ADP leverages aggregated state variables to approximate the value function as a new way, balancing computational efficiency and economy against the large value function space of massive ACs. By training the value function offline using historical data, the TCA-ADP efficiently achieves near-optimal real-time scheduling of massive ACs through parallel and closed-form disaggregation. Case studies demonstrate the effectiveness and scalability of the TCA-ADP, highlighting its aggregation accuracy, uncertainty handling, and the trade-off between economy and tractability.

关键词： Real-time systems Uncertainty Heuristic algorithms Cause effect analysis Training dynamic programming Approximation algorithms Inverters Accuracy Renewable energy sources Air conditioner state variable aggregation real-time scheduling approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Multi-period integrated production and delivery scheduling to enhance online orders fulfillment considering due dates

引用

FLEXIBLE SERVICES AND MANUFACTURING JOURNAL 2025年 1-33页

作者： Wang, Jie Yang, Peng Tsinghua Univ Inst Data & Informat Shenzhen Int Grad Sch Shenzhen 518055 Peoples R China Tsinghua Univ Shenzhen Int Grad Sch Div Logist & Transportat Shenzhen 518055 Peoples R China Purdue Univ Sch Ind Engn W Lafayette IN 47907 USA

Online order fulfillment under limited capacity is essential for e-commerce when faced with demand fluctuations or surges. This paper investigates a multi-period integrated production and delivery scheduling problem for fulfilling online orders with due dates. We model the problem as the Markov decision process and develop an approximate dynamic programming (ADP) algorithm to minimize the weighted sum of average waiting time and order timeouts. The performance and robustness of the ADP strategy are validated through numerical experiments across various scenarios with differing order volumes and demand patterns. We find that for a fixed total volume of orders, different patterns of demand arrival influence the integrated production and delivery schedules, among which scenarios with a decreasing demand pattern over time are particularly challenging. Additionally, we discover that settings with flexible delivery departure times are more advantageous when demand exceeds the predefined production capacities of businesses, whereas fixed delivery departure times do not result in significant negative impacts when demand is relative low. Our findings suggest valuable insights for e-commerce operations regarding adaptable production and delivery schedules in response to varying demand patterns.

关键词： Online order fulfillment Integrated production-delivery scheduling approximate dynamic programming Markov decision process Non-stationary demand

来源：评论

学校读者我要写书评

暂无评论

Lookahead scenario relaxation for dynamic time window assignment in service routing

引用

TRANSPORTATION RESEARCH PART B-METHODOLOGICAL 2025年 192卷

作者： Paradiso, Rosario Roberti, Roberto Ulmer, Marlin Vrije Univ Amsterdam Dept Operat Analyt Amsterdam Netherlands Univ Padua Dept Informat Engn Padua Italy Otto von Guericke Univ Chair Management Sci D-39106 Magdeburg Germany

We consider a problem where customers dynamically request next-day home service, e.g., repair or installments. Unlike attended home delivery, customers cannot select a time window (TW), the service provider assigns a next-day TW to each new customer if the customer can feasibly be inserted in the service route of the next day without violating the TWs of the existing customers. Otherwise, customer service will be postponed to another day (which is outside the scope of this work). The provider aims to serve many customers the next day for fast service and efficient operations. Thus, TWs have to be assigned to keep the flexibility of the fleet for future requests. For such anticipatory assignments, we propose a stochastic lookahead method that samples a set of future request scenarios, solves the corresponding team-orienteering problems with TWs, and uses the solutions to evaluate current TW assignment decisions. For real-time solutions to the team orienteering problem, we propose to approximate its optimal solution value with an upper bound. The bound is obtained by solving the linear relaxation of a set packing reformulation via column generation. We test our algorithm on Iowa City data and compare it to several benchmark policies. The results show that our method significantly increases customer service, and our relaxation is essential for effective decisions. We further show that our policy does not lead to observable discrimination against inconveniently located customers.

关键词： Next-day service dynamic time-window assignment Stochastic lookahead approximate dynamic programming Column generation

来源：评论

学校读者我要写书评

暂无评论

Data-driven optimal shared control of unmanned aerial vehicles

引用

NEUROCOMPUTING 2025年 622卷

作者： Tan, Junkai Xue, Shuangsi Guo, Zihang Li, Huan Cao, Hui Chen, Badong Xi An Jiao Tong Univ Sch Elect Engn Xian 710049 Peoples R China Xi An Jiao Tong Univ Sch Elect Engn State Key Lab Elect Insulat & Power Equipment Xian 710049 Peoples R China Xi An Jiao Tong Univ Natl Engn Res Ctr Visual Informat & Applicat Natl Key Lab Human Machine Hybrid Augmented Intell Xian 710049 Peoples R China Xi An Jiao Tong Univ Inst Artificial Intelligence & Robot Xian 710049 Peoples R China

Cooperation between humans and autonomy is a critical topic of unmanned aerial vehicle (UAV) control. How to co-pilot the UAV with human operator to achieve optimal performance presents a significant challenge. In this paper, we propose a novel data-driven optimal shared control method for UAV using the Koopman operators to predict the nonlinear dynamics of the UAVs. An original shared control mechanism is established to allocate the relationship between optimal and human control inputs. The model of the system is learned from human maneuver data via the Koopman operator approach, and the optimal controller is approximated online using reinforcement learning techniques. The Lyapunov theory analyzes the stability of the proposed method. Compared with offline RL methods, the proposed method can learn the optimal controller online without a precise UAV dynamics model from human maneuver data. The effectiveness of the proposed method is demonstrated by numerical and Human-in-the-loop (HiTL) simulation.

关键词： Shared control Reinforcement learning Optimal control Koopman operator Unmanned aerial vehicle approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

approximate Information States for Worst Case Control and Learning in Uncertain Systems

引用

IEEE TRANSACTIONS ON AUTOMATIC CONTROL 2025年第1期70卷 127-142页

作者： Dave, Aditya Nishanth Venkatesh, S. Malikopoulos, Andreas A. Cornell Univ Sch Civil & Environm Engn Ithaca NY 14850 USA

In this article, we investigate discrete-time decision-making problems in uncertain systems with partially observed states. We consider a nonstochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by using the notion of the information state and approximate information state and introduce conditions to identify an uncertain variable that can be used to compute an optimal strategy through a dynamic program (DP). Next, we relax these conditions and define approximate information states that can be learned from output data without knowledge of system dynamics. We use approximate information states to formulate a DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.

关键词： Costs Decision making Stochastic processes Reinforcement learning Aerospace electronics Vehicle dynamics Uncertain systems approximate dynamic programming robust reinforcement learning uncertain systems worst case control

来源：评论

学校读者我要写书评

暂无评论

Unlocking the value in product return data: Inventory management with sales dependent stochastic product return flows from multiple periods

引用

INTERNATIONAL JOURNAL OF PRODUCTION ECONOMICS 2025年 285卷

作者： Gokbayrak, Esra Kayis, Enis Gullu, Refik Ozyegin Univ Dept Ind Engn Nisantepe MahOrman Sok TR-34794 Istanbul Turkiye Bogazici Univ Dept Ind Engn TR-34342 Istanbul Turkiye

In the fast fashion retail sector, handling product returns has become a significant challenge due to rapidly changing consumer preferences and high product return rates. These retailers are now inclined to consider product return flows in managing product inventories using detailed product return data. This study investigates an optimal inventory control policy for a retailer facing stochastic product returns from multiple previous sale periods to maximize expected profit during a single selling season. The problem is formulated using dynamic programming, and due to its computational complexity, we propose an approximate dynamic programming value iteration algorithm using basis functions. Our proposed algorithm reduces the solution time drastically without a significant sacrifice from optimality. We quantify the value of leveraging detailed return information and demonstrate that our proposed model increases the retailer's profit by 9% in the base case and up to 31% considering other cases compared to a model ignoring such information, especially under decreasing product prices over time or per period order capacity constraints. Finally, using an extensive computational study, we propose managerial insights on how to best leverage the value in the product return data using advanced analytics for fast-fashion retailers.

关键词： Inventory management Stochastic product returns approximate dynamic programming Retail operations

来源：评论

学校读者我要写书评

暂无评论

Explainable and Safety Aware Deep Reinforcement Learning-Based Control of Nonlinear Discrete-Time Systems Using Neural Network Gradient Decomposition

引用

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2025年 22卷 13556-13568页

作者： Farzanegan, Behzad Jagannathan, Sarangapani Missouri Univ Sci & Technol Dept Elect & Comp Engn Rolla MO 65401 USA

This paper presents an explainable deep-reinforcement learning (DRL)-based safety-aware optimal adaptive tracking (SOAT) scheme for a class of nonlinear discrete-time (DT) affine systems subject to state inequality constraints. The DRL-based SOAT utilizes a multilayer neural network (MNN)-based actor-critic to estimate the cost function and optimal policy while the MNN update laws are tuned both using the singular value decomposition (SVD) of activation function gradient in order to mitigate the vanishing gradient issue and safety-aware Bellman error at each layer. An approximate safety-aware optimal policy is developed using Karush-Kuhn-Tucker (KKT) conditions by incorporating the higher-order control barrier function (HOCBF) into the Hamiltonian through the Lagrangian multiplier. The resulting safety-aware Bellman error helps with safe exploration both during online learning phase and at steady state without any explicit actor-critic MNN update law changes. To study the explainability and gain insights, we employ the Shapley Additive Explanations (SHAP) method to construct an explainer model for the DRL-based SOAT scheme in order to identify the important features in determining the optimal policy. The overall stability is established. Finally, the effectiveness of the proposed method is demonstrated on Shipboard Power Systems (SPS), achieving over a 35% reduction in cumulative cost compared to the existing actor-critic MNN optimal control policy. Note to Practitioners-In practical control systems, meeting safety constraints is often critical since ignoring constraints can lead to degraded performance or damage to equipment. This paper addresses the challenge of a safe DRL-based control approach that not only optimizes performance but also integrates robust safety assurances. Our DRL-based SOAT scheme specifically targets nonlinear discrete-time systems that must satisfy state inequality constraints. The successful proposed control performance in simulations on a

关键词： Safety Multi-layer neural network Optimal control Artificial neural networks Trajectory Cost function Tuning Computational modeling Training Steady-state Deep reinforcement learning approximate dynamic programming singular value decomposition-based weight tuning online safety-aware control explainability

来源：评论

学校读者我要写书评

暂无评论

Markov-Decision-Process-Based Reinforcement Learning for Emergency Vehicles of Dispatch and Reallocation Strategies through the Lens of Efficiency and Fairness: Evidence from Chinese Emergency Medical Services System

引用

TRANSPORTATION RESEARCH RECORD 2025年第4期2679卷 1015-1030页

作者： Luan, Siliang Pan, Fuquan Chen, Xiufeng Bing, Qichun Yang, Jinshun Jin, Xin Qingdao Univ Technol Sch Civil Engn Qingdao Peoples R China Minist Educ Engn Res Ctr Concrete Technol Marine Environm Qingdao Peoples R China

Efficient proactive emergency vehicle planning is crucial for effective emergency response management systems. This paper develops a flexible deep reinforcement learning framework to address uncertainties in dynamic emergency requests and complex traffic conditions. The Markov decision process model is proposed to optimize ambulance dispatch and reallocation, focusing on minimizing average response time, reducing delays from traffic congestion and patient severity misclassification, and enhancing general fairness. To tackle the challenges of the unbounded MDP state space, we employ a least-squares-based approximate policy iteration model for the upper bound and a linear programming model for the lower bound. We use Chinese emergency medical services system data for computational experiments and select average response time, fraction of delay, risk level, and Gini coefficient to evaluate the performance of our model. The numerical results demonstrate our optimal flexible policy outperforms the first-in-first-served rule, improving both efficiency and equity in medical decision-making.

关键词： emergency medical service emergency vehicle operations Markov decision process dispatch and relocation approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：