检索结果-内蒙古大学图书馆

Beyond stochastic dynamic programming: a heuristic sampling method for optimizing conservation decisions in very large state spaces

引用

METHODS IN ECOLOGY AND EVOLUTION 2011年第2期2卷 221-228页

作者： Nicol, Sam Chades, Iadine Univ Queensland Sch Math & Phys Ctr Ecol St Lucia Qld 4072 Australia CSIRO Sustainable Ecosyst St Lucia Qld 4067 Australia

P>1. When managing endangered species the consequences of making a poor decision can be extinction. To make a good decision, we must account for the stochastic dynamic of the population over time. To this end stochastic dynamic programming (SDP) has become the most widely used tool to calculate the optimal policy to manage a population over time and under uncertainty. 2. However, as a result of its prohibitive computational complexity, SDP has been limited to solving small dimension problems, which results in SDP models that are either oversimplified or approximated using greedy heuristics that only consider the immediate rewards of an action. 3. We present a heuristic sampling (HS) method that approximates the optimal policy for any starting state. The method is attractive for problems with large state spaces as the running time is independent of the size of the problem state space and improves with time. 4. We demonstrate that the HS method out-performs a commonly used greedy heuristic and can quickly solve a problem with 33 million states. This is roughly 3 orders of magnitude larger than the largest problems that can currently be solved with SDP methods. 5. We found that HS out-performs greedy heuristics and can give near-optimal policies in shorter timeframes than SDP. HS can solve problems with state spaces that are too large to optimize with SDP. Where the state space size precludes SDP, we argue that HS is the best technique.

关键词： Leipoa ocellata Markov decision processes metapopulation on-line sparse sampling algorithm optimal management stochastic dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Conservation decision-making in large state spaces

引用

ECOLOGICAL MODELLING 2010年第21期221卷 2531-2536页

作者： Nicol, Samuel C. Chades, Ladine Linke, Simon Possingham, Hugh P. Univ Queensland Spatial Ecol Lab St Lucia Qld 4072 Australia CSIRO Sustainable Ecosyst St Lucia Qld 4067 Australia Griffith Univ Australian Rivers Inst Nathan Qld 4111 Australia

When looking for the best course of management decisions to efficiently conserve metapopulation systems, a classic approach in the ecology literature is to model the optimisation problem as a Markov decision process and find an optimal control policy using exact stochastic dynamic programming techniques. Stochastic dynamic programming is an iterative procedure that seeks to optimise a value function at each timestep by evaluating the benefits of each of the actions in each state of the system defined in the Markov decision process. Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal. We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the

关键词： Markov decision processes Stochastic dynamic programming Metapopulation Optimal management on-line sparse sampling algorithm

来源：评论

学校读者我要写书评

暂无评论

Conservation decision-making in large state spaces 18

Conservation decision-making in large state spaces

引用

IMACS World Congress/Modelling and Simulation Society-of-Australia-and-New-Zealand (MSSANZ)/18th MODSIM09 Biennial Conference on Modelling and Simulation

作者： Nicol, S. C. Chades, I. Linke, S. Possingham, H. P. Univ Queensland Brisbane Qld Australia

ISBN: (纸本)9780975840078

For metapopulation management problems with small state spaces, it is typically possible to model the problem as a Markov decision process (MDP), and find an optimal control policy using stochastic dynamic programming (SDP). SDP is an iterative procedure that seeks to optimise a value function at each timestep by trying each of the actions defined in the MDP. Although SDP gives the optimal solution to conservation management questions in a stochastic world, its applicability has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. A large state space makes optimal SDP solutions computationally expensive to compute because optimal SDP techniques require the value function to be updated for the entire state space for every time step. The high computational requirements of large SDP problems means that only simple population management problems can be analysed. In this paper we present an application of the on-line sparse sampling algorithm proposed by Kearns, Mansour & Ng (2002), which can be used to approximate the optimal solution of a MDP for a given starting state. The algorithm is particularly attractive for problems with large state spaces as it has a running time that is independent of the size of the state space of the problem. We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation where we define the management objective to be to maximise the number of occupied patches during the management horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that the approximation algorithm out

关键词： Markov decision processes Stochastic dynamic programming Metapopulation Optimal management on-line sparse sampling algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：