检索结果-内蒙古大学图书馆

Policy Rollout Action Selection in Continuous Domains for Sensor Path Planning

IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS 2021年第4期57卷 2247-2264页

作者： Hoffmann, Folker Charlish, Alexander Ritchie, Matthew Griffiths, Hugh Fraunhofer FKIE D-53343 Wachtberg Germany UCL London WC1E 6BT England

Policy rollout is a method for the online computation of future costs in approximate dynamic programming and has been utilized for various problems, including sensor management. In previous work, it has predominately been applied to the selection of actions from discrete sets. In this article, we present methods for action selection from continuous sets and analyze their tradeoffs. The methods are evaluated on the problem of sensor path planning, with the intent of minimizing the time to localize an emitter using bearing measurements.

关键词： Path planning Uncertainty Time measurement Monte Carlo methods Markov processes Unmanned aerial vehicles Random variables approximate dynamic programming emitter localization Markov decision process (MDP) partially observable Markov decision process (POMDP) policy rollout sensor management sensor path planning stochastic optimization

来源：评论

学校读者我要写书评

暂无评论

Warehousing platform's revenue management: A dynamic model of coordinating space allocation for self-use and rent

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2021年第1期293卷 167-176页

作者： Shi, Ye Yu, Yugang Dong, Yuxuan Univ Sci & Technol China Sch Management 96 Jinzhai Rd Hefei 230026 Anhui Peoples R China Shanghai Jiao Tong Univ Sino US Global Logist Inst 1954 Huashan Rd Shanghai 200030 Peoples R China

Recently, some giant retail platforms have begun to invest large amounts of money in building warehouses. With abundant warehousing space, these platforms are renting out space to third-party retailers (TPRs) with insufficient storage space. However, such rental programs generate a new operational problem for these platforms, namely, how to divide their space between space for self-use and space for rent. To address the optimal space allocation decisions for this operational problem, we study a finite horizon and periodic review warehouse model where some items are sold by the platform, and other items are sold by TPRs. More specifically, we start with a base model with two items and prove the optimality of a base stock policy for the base model and the monotonicity of the optimal space allocation decisions regarding several key system parameters (such as inventory and capacity parameters). We then study several extensions of the base model by considering demand forecasts, demand correlation, and multiple items (more than two). Particularly, when addressing the extension with multiple items, we propose a heuristic based on the idea of approximate dynamic programming to overcome the curse of dimensionality. To examine the effectiveness of our heuristic, we present a computational study that is based on real-life datasets provided by the largest furniture retail platform in China (Red-Star Macalline). The numerical results demonstrate that our heuristic has the potential to solve medium- and large-scale problems effectively and efficiently. (C) 2020 Elsevier B.V. All rights reserved.

关键词： E-commerce Space allocation Rent Warehouse revenue management approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Managing Shutdown Decisions in Merchant Commodity and Energy Production: A Social Commerce Perspective

引用

M&SOM-MANUFACTURING & SERVICE OPERATIONS MANAGEMENT 2021年第2期23卷 311-330页

作者： Trivella, Alessio Nadarajah, Selvaprabu Fleten, Stein-Erik Mazieres, Denis Pisinger, David Swiss Fed Inst Technol Inst Transport Planning & Syst CH-8092 Zurich Switzerland Univ Illinois Coll Business Adm Chicago IL 60607 USA Norwegian Univ Sci & Technol Dept Ind Econ & Technol Management N-7491 Trondheim Norway Birkbeck Univ London Dept Econ Math & Stat London WC1E 7HX England Tech Univ Denmark Dept Management Engn DK-2800 Lyngby Denmark

Problem definition: Merchant commodity and energy production assets operate in markets with volatile prices and exchange rates. Plant closures adversely affect societal entities beyond the specific plant being shut down, such as the parent company and the local community. Motivated by an aluminum producer, we study if mitigating these hard-to-assess broader impacts of a shutdown is financially viable using the plant's operating flexibility. Academic/practical relevance: Our social commerce perspective toward managing shutdown decisions deviates from the commonly used asset value maximization objective in merchant operations. Identifying operating policies that delay or decrease the likelihood of a shutdown without incurring a significant asset value loss supports socially responsible plant shutdown decisions. Methodology: We formulate a constrained Markov decision process to manage shutdown decisions and limit the probability of future plant closures. We provide theoretical support for approximating this intractable model using unconstrained stochastic dynamic programs with modified shutdown costs and explore two classes of operating policies. Our first policy leverages anticipated regret theory, and the second policy generalizes, using machine learning, production-margin heuristics used in practice. We compute the former and latter policies using a least squares Monte Carlo method and combining this method with binary classification, respectively. Results: Anticipated-regret policies possess desirable asymptotic properties absent in classification-based policies. On instances created using real data, anticipated-regret and classification-based policies outperform practice-based production-margin strategies. Significant reductions in shutdown probability and delays in plant closures are possible while incurring small asset value losses. Managerial implications: A plant's operating flexibility provides an effective lever to balance the social objective to reduce clos

关键词： merchant production energy and commodity operations socially responsible operations approximate dynamic programming least squares Monte Carlo shutdown decisions

来源：评论

学校读者我要写书评

暂无评论

Robust Reinforcement Learning with Diffusion Wavelets

Robust Reinforcement Learning with Diffusion Wavelets

引用

作者： Seyedmazloom, Ali George Mason University

学位级别：Ph.D., Doctor of Philosophy

Reinforcement Learning is a method of learning from the environment by constantly observing it and evaluating its response to a set of actions. Long-term learning of a dynamic system for aforementioned interactions where its constituents evolve temporally, accumulates valuable knowledge about system’s history and actions taken. Reinforcement Learning (RL) takes advantage of this gathered knowledge to identify an optimal policy that essentially dictates which decision(s) should be made when the system is in a certain state in order to guide it to achieve the best objective value in the long run. Since system’s objective value is nothing but accumulated discounted rewards over various system states, it is crucial to define and retrieve rewards from the environment as genuinely as possible. However, there are several instances where we might be receiving noisy, corrupt, or even intentionally perturbed state perceptions that contain distorted rewards. In robotics for example, it is quite common to receive noisy feedback through faulty sensors or adverse environment because of rain, wind, lack of light, etc. The focus of this research is to study and utilize methods to properly handle intentionally corrupted discrete-state perceptions and their associated reward channel that are crafted in a way which will increase the likelihood of the model taking actions not in line with its genuine objectives but of an unknown adversary’s. This may seriously compromise RL model’s functionality and addressing it, is of utmost importance in Secure Machine Learning domain as it has been the case with Artificial Neural Network models targeted by the “Adversarial Attacks”. A value-based RL model needs to assess all possible decisions based on the quality of the upcoming states and these quality values (known as q-values) should be obtained from a robust process. While many state-of-the-art RL models use Artificial Neural Networks for providing state values to the RL agent, in this researc

关键词： approximate dynamic programming Diffusion wavelets Operations research Reinforcement learning Robust learning

来源：评论

学校读者我要写书评

暂无评论

Discrete-Time Non-Zero-Sum Games With Completely Unknown dynamics

引用

IEEE TRANSACTIONS ON CYBERNETICS 2021年第6期51卷 2929-2943页

作者： Song, Ruizhuo Wei, Qinglai Zhang, Huaguang Lewis, Frank L. Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Peoples R China Univ Texas Arlington UTA Res Inst Ft Worth TX 76118 USA

In this article, off-policy reinforcement learning (RL) algorithm is established to solve the discrete-time N-player nonzero-sum (NZS) games with completely unknown dynamics. The N-coupled generalized algebraic Riccati equations (GARE) are derived, and then policy iteration (PI) algorithm is used to obtain the N-tuple of iterative control and iterative value function. As the system dynamics is necessary in PI algorithm, off-policy RL method is developed for discrete-time N-player NZS games. The off-policy N-coupled Hamilton-Jacobi (HJ) equation is derived based on quadratic value functions. According to the Kronecker product, the N-coupled HJ equation is decomposed into unknown parameter part and the system operation data part, which makes the N-coupled HJ equation solved independent of system dynamics. The least square is used to calculate the iterative value function and N-tuple of iterative control. The existence of Nash equilibrium is proved. The result of the proposed method for discrete-time unknown dynamics NZS games is indicated by the simulation examples.

关键词： Adaptive critic designs adaptive dynamic programming approximate dynamic programming discrete-time nonzero-sum (NZS) off-policy reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Electric Vehicle Routing with Public Charging Stations

引用

TRANSPORTATION SCIENCE 2021年第3期55卷 637-659页

作者： Kullman, Nicholas D. Goodson, Justin C. Mendoza, Jorge E. Univ Tours CNRS LIFAT EA 6300 ROOT ERL CNRS 7002 F-37200 Tours France St Louis Univ Richard A Chaifetz Sch Business St Louis MO 63103 USA HEC Montreal Montreal PQ H3T 2A7 Canada Ctr Interuniv Rech Reseaux Entreprise Logist & Tr Montreal PQ H3T 1J4 Canada

We introduce the electric vehicle routing problem with public-private recharging strategy in which vehicles may recharge en route at public charging infrastructure as well as at a privately-owned depot. To hedge against uncertain demand at public charging stations, we design routing policies that anticipate station queue dynamics. We leverage a decomposition to identify good routing policies, including the optimal static policy and fixed-route-based rollout policies that dynamically respond to observed queues. The decomposition also enables us to establish dual bounds, providing a measure of goodness for our routing policies. In computational experiments using real instances from industry, we show the value of our policies to be within 10% of a dual bound. Furthermore, we demonstrate that our policies significantly outperform the industry-standard routing strategy in which vehicle recharging generally occurs at a central depot. Our methods stand to reduce the operating costs associated with electric vehicles, facilitating the transition from internal-combustion engine vehicles.

关键词： dynamic vehicle routing electric vehicles fixed routes information relaxation information penalties approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Managing uncertainty in demand through inventory and workforce development

引用

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH 2021年第21期59卷 6507-6527页

作者： Valeva, Silviya Hewitt, Mike Thomas, Barrett W. Rice Univ George R Brown Sch Engn Dept Computat & Appl Math Houston TX 77005 USA Loyola Univ Quinlan Sch Business Informat Syst & Supply Chain Management Dept Chicago IL 60611 USA Univ Iowa Tippie Coll Business Dept Business Analyt Iowa City IA USA

This paper studies whether and how a manufacturer can mitigate the effects of demand uncertainty during production ramp up by jointly managing inventory and capacity development, determined by the productivity of a heterogeneous workforce. One way the manufacturer can direct its capacity development is by recognising that workers become more productive as they gain experience when making scheduling decisions. We consider a situation in which demands for multiple products are periodically revealed and a manufacturer must decide what each individual does in the production process, how much inventory is accumulated, and how demands are met. To study the potential benefits, we formulate a Markov decision process model which we solve using a one-step lookahead approximation. Results of an extensive computational study show that, given the effectiveness associated with both directing capacity and holding inventory, manufacturers should consider the workforce's capacity to learn when making job assignments during production ramp up. Analysis of the results suggests that inventory and capacity building patterns depend on product value and the workforce's experience levels. Further analysis shows that slower learners and those with the potential to achieve higher productivity should specialise more. Faster learners, on the other hand, are best suited for cross training.

关键词： Workforce planning inventory management capacity development learning ramp up approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

dynamic Repair Scheduling for Transmission Systems Based on Look-Ahead Strategy Approximation

引用

IEEE TRANSACTIONS ON POWER SYSTEMS 2021年第4期36卷 2918-2933页

作者： Yan, Jiahao Hu, Bo Xie, Kaigui Niu, Tao Li, Chunyan Tai, Heng-Ming Chongqing Univ State Key Lab Power Transmiss Equipment & Syst Se Chongqing 400030 Peoples R China Univ Tulsa Dept Elect & Comp Engn Tulsa OK 74104 USA

This paper intends to address the dynamic repair scheduling of electric power transmission systems based on look-ahead strategy approximation. The objective is to minimize system functionality loss during the restoration stage after disruptive events. A series of decisions regarding which damaged component to be repaired has to be made successively considering currently available information of repair time and its uncertainty in the future. To achieve this goal, the dynamic repair scheduling problem is represented as a stochastic Markovian decision process (MDP). To overcome the computational complexity of MDP derived from exponentially growing state space, the cost-to-go function is approximated by a look-ahead strategy based on repair importance ordering. Stage-dependent coefficients are used to balance the approximated functionality loss at different decision stages. The tradeoff between the efficiency and optimality can be achieved by adjusting the look-ahead depth and the updating policy of look-ahead strategy. The IEEE-14 and 118-bus systems were used for performance evaluation of the proposed method and comparison with various approaches. The results show that it can produce decisions close to the best-known solutions within small amount of time.

关键词： Maintenance engineering Task analysis Indexes Power transmission lines Schedules dynamic scheduling Generators Transmission restoration dynamic repair scheduling look-ahead strategy approximate dynamic programming repair importance ordering

来源：评论

学校读者我要写书评

暂无评论

dynamic Capacity Allocation for Elective Surgeries: Reducing Urgency-Weighted Wait Times

引用

M&SOM-MANUFACTURING & SERVICE OPERATIONS MANAGEMENT 2021年第2期23卷 407-424页

作者： Carew, Stephanie Nagarajan, Mahesh Shechter, Steven Arneja, Jugpal Skarsgard, Erik Univ British Columbia Sauder Sch Business Vancouver BC V6T 1Z2 Canada Univ British Columbia Fac Med Vancouver BC V5Z 1M9 Canada

Problem definition: Given the variety of urgency levels in highly utilized operating rooms, capacity allocation decisions can have a major impact on how wait times are rationed. We examine a longer-term sequential capacity planning problem in which a hospital allocates operating room time to different surgical specialties. We seek to minimize an urgency-weighted wait-time metric. Academic/practical relevance: Our data set on patient selection patterns revealed considerable noise in the queuing discipline. We apply an urn model to generate a probabilistic queuing discipline, which validates well against the selection patterns observed in practice. We believe that this model may prove to be useful for representing noisy queuing disciplines in other settings. Also, our validated simulation model, in combination with our proposed solution approach, demonstrates a substantial reduction in urgency-weighed wait times. Methodology: For representing the noisy queuing discipline, we fit a Wallenius noncentral hypergeometric distribution. We formulate the capacity allocation problem as a Markov decision process. The large state space and detailed system dynamics lead us to simulation-based dynamic programming approaches for finding good capacity allocation decisions. Rather than approximate the expected cost-to-go function, we propose a limited look-ahead policy and embed this in a rolling-horizon framework. Results: Our baseline model-based allocation policy yields a 14.3% reduction in urgency-weighed wait time compared with current practice. It also results in a 21.0% improvement in the number of patients treated within their urgency-based recommended wait-time limits. Managerial implications: In elective surgery settings, it may be important to ration capacity in a way that considers the different urgency levels of patients. We propose a flexible modeling approach for achieving this.

关键词： surgical waiting list dynamic programming approximate dynamic programming

来源：评论

学校读者我要写书评

暂无评论

A Stochastic Spatiotemporal Decomposition Decision-Making Approach for Real-Time dynamic Energy Management of Multi-Microgrids

引用

IEEE TRANSACTIONS ON SUSTAINABLE ENERGY 2021年第2期12卷 821-833页

作者： Mo, Xiemin Zhu, Jianquan Chen, Jiajun Guo, Ye Xia, Yunrui Liu, Mingbo South China Univ Technol Sch Elect Power Engn Guangzhou 510640 Peoples R China

This paper studies the real-time dynamic energy management (DEM) of multi microgrids (MMGs) considering active and reactive power flow constraints, voltage constraints, battery operational characters, and uncertainties in the renewable generation and load. A stochastic spatiotemporal decomposition decision-making framework is proposed based on approximate dynamic programming (ADP) to make decentralized decisions in both spatial and temporal dimensions. The tie-line power and the state of charge (SOC) of the battery are coordinated in real time to deal with the uncertainties in the upcoming future, preserving the decision independence of MGs and periods. And the shift factors are derived to consider active and reactive power flow constraints of MMGs in the stochastic spatiotemporal decomposition framework. Moreover, the historical information is utilized offline to avoid the dependency on forecast information and iterative calculation, while near-optimal solutions can be obtained in real time. Case studies on several MMG test systems and a real MMG system demonstrate the effectiveness of the proposed approach.

关键词： Spatiotemporal phenomena Batteries State of charge Reactive power Real-time systems Load flow Decision making Multi microgrids spatiotemporal decomposition approximate dynamic programming uncertainty power flow constraint shift factor method

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：