检索结果-内蒙古大学图书馆

Structured threshold policies for dynamic sensor scheduling - A partially observed Markov decision process approach

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2007年第10期55卷 4938-4957页

作者： Krishnamurthy, Vikram Djonin, Dejan V. Univ British Columbia Dept Elect & Comp Engn Vancouver BC V6T 1Z4 Canada Univ British Columbia Dyaptive Syst Inc Dept Elect & Comp Engn Vancouver BC V6T 1Z4 Canada

We consider the optimal sensor scheduling problem formulated as a partially observed Markov decision process (POMDP). Due to operational constraints, at each time instant, the scheduler can dynamically select one out of a finite number of sensors and record a noisy measurement of an underlying Markov chain. The aim is to compute the optimal measurement scheduling policy, so as to minimize a cost function comprising of estimation errors and measurement costs. The formulation results in a nonstandard POMDP that is nonlinear in the information state. We give sufficient conditions on the cost function, dynamics of the Markov chain and observation probabilities so that the optimal scheduling policy has a threshold structure with respect to a monotone likelihood ratio (MLR) ordering. As a result, the computational complexity of implementing the optimal scheduling policy is inexpensive. We then present stochastic approximation algorithms for estimating the best linear MLR order threshold policy.

关键词： bayesian filtering monotone likelihood ratio (MLR) ordering partially observed Markov decision processes (POMDPs) sensor scheduling stochastic approximation algorithms stochastic dynamic programming threshold policies

来源：评论

学校读者我要写书评

暂无评论

stochastic discrete-time Nash games with constrained state estimators

引用

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS 2002年第1期114卷 171-188页

作者： Kian, AR Cruz, JB Simaan, MA Ohio State Univ Dept Elect Engn Columbus OH 43210 USA Univ Pittsburgh Dept Elect Engn Pittsburgh PA USA

In this paper, we consider stochastic linear-quadratic discrete-time Nash games in which two players have access only to noise-corrupted output measurements. We assume that each player is constrained to use a linear Kalman filter-like state estimator to implement his optimal strategies. Two information structures available to the players in their state estimators are investigated. The first has access to one-step delayed output and a one-step delayed control input of the player. The second has access to the current output and a one-step delayed control input of the player. In both cases, statistics of the process and statistics of the measurements of each player are known to both players. A simple example of a two-zone energy trading system is considered to illustrate the developed Nash strategies. In this example, the Nash strategies are calculated for the two cases of unlimited and limited transmission capacity constraints.

关键词： stochastic linear-quadratic systems nonzero-sum discrete-time Nash games optimal Nash strategies estimator-controller stochastic dynamic programming Nash equilibrium

来源：评论

学校读者我要写书评

暂无评论

An Integer programming Approach for Evaluating R&D Funding Decisions With Optimal Budget Allocations

引用

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT 2012年第4期59卷 679-691页

作者： Eckhause, Jeremy M. Gabriel, Steven A. Hughes, Danny R. Logist Management Inst Mclean VA 22102 USA Univ Maryland Dept Civil & Environm Engn College Pk MD 20742 USA Univ Oklahoma Dept Hlth Adm & Policy Oklahoma City OK 73019 USA

The use of real options techniques for research and development (R&D) project selection to mitigate the uncertainties has been shown to increase overall project value. While recent approaches employing stochastic dynamic programs (SDP) produce optimal solutions for many applications, this approach does not easily accommodate the inclusion of an optimal a priori budget allocation or side constraints, since their formulations are scenario specific. We formulate an integer program (IP) whose solution is equivalent to previous SDP real options models but facilitates the incorporation of these additional features and may be solved using commercial IP solvers. This IP formulation can solve what would otherwise be a nested two-level problem where the lower level problem is an SDP. We then compare the performance of the IP to that received by the SDP using a case study from the literature.

关键词： Integer programming (IP) project selection real options research and development (R&D) portfolios stochastic dynamic programming two-level problems

来源：评论

学校读者我要写书评

暂无评论

Fixed versus flexible production systems: A real options analysis

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2008年第1期188卷 169-184页

作者： Fontes, Dalila B. M. M. Univ Porto Fac Econ LIACC P-4200464 Oporto Portugal

In this work, we address investment decisions in production systems by using real options. As is standard in literature, the stochastic variable is assumed to be normally distributed and then approximated by a binomial distribution, resulting in a binomial lattice. The methodology establishes a discrete-valued lattice of possible future values of the underlying stochastic variable (demand in our case) and then, computes the project value. We have developed and implemented stochastic dynamic programming models both for fixed and flexible capacity systems. In the former case, we consider three standard options: the option to postpone investment, the option to abandon investment, and the option to temporarily shut-down production. For the latter case, we introduce the option of corrective action, in terms of production capacity, that the management can take during the project by considering the existence of one of the following: (i) a capacity expansion option;(ii) a capacity contraction option;or (iii) an option considering both expansion and contraction. The full flexible capacity model, where both the contraction and expansion options exist, leads, as expected, to a better project predicted value and thus, investment policy. However, we have also found that the capacity strategy obtained from the flexible capacity model, when applied to specific demand data series, often does not lead to a better investment decision. This might seem surprising, at first, but it can be explained by the inaccuracy of the binomial model. The binomial model tends to undervalue future decreases in the stochastic variable (demand), while at the same time tending to overvalue an increase in future demand values. (C) 2007 Elsevier B.V. All rights reserved.

关键词： fixed and flexible capacity investments real options stochastic dynamic programming binomial lattice investment analysis

来源：评论

学校读者我要写书评

暂无评论

On the value and price-responsiveness of ramp-constrained storage

引用

ENERGY CONVERSION AND MANAGEMENT 2013年 76卷 472-482页

作者： Faghih, Ali Roozbehani, Mardavij Dahleh, Munther A. MIT Elect Engn & Comp Sci EECS Dept Cambridge MA 02139 USA MIT Lab Informat & Decis Syst Cambridge MA 02139 USA

The primary concerns of this paper are twofold: understanding the value of storage in the presence of ramp constraints and exogenous energy prices, and understanding the implications of the associated optimal storage management policy for qualitative and quantitative characteristics of storage response to real-time prices. The optimal policy, along with the associated finite-horizon time-averaged value of storage, are analytically characterized in this paper. An analytical upper bound on the infinite-horizon time-averaged value of storage is also derived. This bound is valid for any achievable realization of prices when the support of the distribution is fixed, and highlights the dependence of the value of storage on ramp constraints and storage capacity. It is shown that while the value of storage is a non-decreasing function of price volatility, due to the finite ramp rate, the value of storage saturates quickly as the capacity increases, regardless of volatility. To study the implications of the optimal policy, computational experiments are presented that suggest optimal utilization of storage can, in expectation, induce a considerable amount of price elasticity near the average price. Then, a computational framework is presented for characterization of the behavior of storage as a function of price and the state of charge, which illustrates a steep buy/sell phase transition in the price-state plane. (C) 2013 Elsevier Ltd. All rights reserved.

关键词： Ramp-constrained energy storage stochastic dynamic programming Control systems

来源：评论

学校读者我要写书评

暂无评论

Modelling and solving resource allocation problems via a dynamic programming approach

引用

INTERNATIONAL JOURNAL OF CONTROL 2021年第6期94卷 1544-1555页

作者： Forootani, Ali Tipaldi, Massimo Zarch, Majid Ghaniee Liuzza, Davide Glielmo, Luigi Univ Sannio Dept Engn I-82100 Benevento Italy Univ Sci & Technol Dept Elect Engn Tehran Iran

In this paper, resource allocation problems are formulated via a set of parallel birth-death processes (BDP). This way, we can model the fact that resources can be allocated to customers at different prices, and that customers can hold them as long as they like. More specifically, a discretisation approach is applied to model resource allocation problems as a set of discrete-time BDPs, which are then integrated into one Markov decision process. The stochastic dynamics of the resulting system are also investigated. As a result, revenue management becomes a stochastic decision-making problem, where price managers can propose suitable prices to the allocation requests such that the maximum expected total revenue is obtained at the end of a predefined finite time horizon. stochastic dynamic programming is employed to solve the related optimisation problem with the support of an ad-hoc Matlab-based application. Several simulations are performed to prove the effectiveness of the proposed model and the optimisation approach.

关键词： Resource allocation problems stochastic dynamic programming Markov chain Markov decision process birth-death process

来源：评论

学校读者我要写书评

暂无评论

Conservation decision-making in large state spaces

引用

ECOLOGICAL MODELLING 2010年第21期221卷 2531-2536页

作者： Nicol, Samuel C. Chades, Ladine Linke, Simon Possingham, Hugh P. Univ Queensland Spatial Ecol Lab St Lucia Qld 4072 Australia CSIRO Sustainable Ecosyst St Lucia Qld 4067 Australia Griffith Univ Australian Rivers Inst Nathan Qld 4111 Australia

When looking for the best course of management decisions to efficiently conserve metapopulation systems, a classic approach in the ecology literature is to model the optimisation problem as a Markov decision process and find an optimal control policy using exact stochastic dynamic programming techniques. stochastic dynamic programming is an iterative procedure that seeks to optimise a value function at each timestep by evaluating the benefits of each of the actions in each state of the system defined in the Markov decision process. Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal. We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the

关键词： Markov decision processes stochastic dynamic programming Metapopulation Optimal management On-line sparse sampling algorithm

来源：评论

学校读者我要写书评

暂无评论

Optimal production and admission control for a stochastic SOM system with demands for product and PSS

引用

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH 2013年第23-24期51卷 7270-7288页

作者： Wang, Kangzhou Jiang, Zhibin Li, Na Geng, Na Shanghai Jiao Tong Univ Sino US Global Logist Inst Shanghai 200030 Peoples R China Shanghai Jiao Tong Univ Sch Mech Engn Dept Ind Engn & Logist Management Shanghai 200030 Peoples R China Shanghai Jiao Tong Univ Res Ctr Serv Sci & Enterprise Innovat Shanghai 200030 Peoples R China

In decades, many traditional manufacturing firms are making the transition from products manufacturers to the providers of product-service systems (PSSs). The new business mode to support this transition is termed service-oriented manufacturing (SOM) paradigm by researchers and practitioners. In this paper, we address the problem of optimal control for a SOM system in which one factory manufactures products in the first stage and then one service centre realises PSSs by adding some product-based services on the products in the second stage. There are two-class external demands: demands for products and PSSs. The PSSs orders have higher priority and are fully accepted whereas the product orders can either be accepted or rejected. The optimal dynamic admission control of the product orders and the optimal dynamic production control of products are examined simultaneously. We first model the system in the context of a simple production-inventory-queue system and formulate the optimal integrated control problem as a continuous Markov decision process in which the objective is to maximise the profit of the firm. Then we characterise the optimal integrated dynamic control by two state-dependent threshold functions. The monotonicity of two optimal control curves verifies that, to achieve an effective management for the SOM system, the controls of manufacturing subsystem and service subsystem in the SOM system must be coo. We also illustrate the impacts of the revenues of both product orders and PSS orders on the optimal policy numerically. It is interesting that the optimal control policy is independent of the revenue of PSS order.

关键词： service-oriented manufacturing inventory control yield management stochastic dynamic programming Markov decision process

来源：评论

学校读者我要写书评

暂无评论

Improved delivery policies for future drone-based delivery systems

引用

EUROPEAN JOURNAL OF OPERATIONAL RESEARCH 2021年第3期294卷 1181-1201页

作者： Chen, Heng Hu, Zhangchen Solak, Senay Univ Nebraska Coll Business Lincoln NE 68583 USA Univ Massachusetts Isenberg Sch Management Amherst MA 01003 USA

It is expected that commercial use of drones in the near future will involve delivery service operations by e-commerce companies. We consider relevant strategic and tactical decisions that these retailers will face in drone-based delivery operations, and derive policies on when to offer drone delivery, what delivery capacity to maintain, and what amount to charge for such deliveries. To this end, we develop a Markov decision process (MDP) framework, and introduce two heuristic procedures, through which near-optimal closed-form solutions can be obtained. The results are aimed at helping online retailers to determine in real time whether and to what extent to offer drone-based delivery for given product categories in different service zones. In addition, we study delivery fee structures and identify drone-based delivery pricing strategies under two widely used delivery pricing schemes. For capacity planning decisions, we describe an algorithm to identify the fleet size to utilize to fulfill uncertain demand in a given service region. We also identify structural characteristics on how these decisions and the expected profit are affected by changes in various problem parameters, which can generate generic insights on drone-based delivery operations for e-commerce companies. We find that retailers should prioritize more profitable items when allocating drone delivery capacity, and invest in adding more drones when per order opportunity costs are higher and promised delivery time thresholds are shorter. Retailers can potentially boost their net profits by increasing the effective promised delivery time threshold and/or decreasing the effective delivery delay costs and per order opportunity costs. (c) 2021 Elsevier B.V. All rights reserved.

关键词： E-commerce Drone-based delivery Capacity management Delivery fee stochastic dynamic programming

来源：评论

学校读者我要写书评

暂无评论

An economic analysis of ecological monitoring

引用

ECOLOGICAL MODELLING 2005年第3-4期189卷 241-250页

作者： White, B Univ Western Australia Sch Agr & Resource Econ Perth WA 6009 Australia

A large number of schemes exist around the world to conserve or establish target natural vegetation communities. These include voluntary agri-environmental contracts, which aim to establish a target vegetation community by compensating farmers to reduce stocking rates. These contracts initiate a stochastic vegetation succession which increases the probability of establishing the target vegetation community. To ensure the scheme is achieving its objectives, regulators monitor the vegetation succession and decide whether a contract should stop or continue. If vegetation succession can be represented by a Markov chain, the regulator's problem of when to monitor, the best monitoring method and when to stop or continue a contract can be solved by a partially observed Markov decision process (POMDP). The results, for the conservation and restoration of heather Moorland in the Cambrian Mountains of Wales, show that the frequency and quality of monitoring depends upon monitoring costs and the regulators prior probabilities for the vegetation state. (c) 2005 Elsevier B.V. All fights reserved.

关键词： environmental monitoring stochastic dynamic programming POMDP Markov chain

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：