In this paper, the problem of state tracking with controlled observations is considered for a system modeled by a discrete-time, finite-state Markov chain. The system state is 'hidden' and observed via conditi...
详细信息
ISBN:
(纸本)9781479903573
In this paper, the problem of state tracking with controlled observations is considered for a system modeled by a discrete-time, finite-state Markov chain. The system state is 'hidden' and observed via conditionally Gaussian measurements that are shaped by the underlying state and an exogenous control input. Following an innovations approach, a Kalman-like filter is derived to estimate the Markov chain system state. To optimize the control strategy, the associated mean-squared error is used as an optimization criterion for a partially observable Markov Decision Process (POMDP). The optimal solution is determined via stochastic dynamic programming. Numerical results are presented for the application of physical activity detection in heterogeneous, wireless body area networks.
The optimal portfolio problem for a bank account, single risky stock and a rolling horizon bond is developed. The stochastic short-term interest rate with the Cox-Ingersoll-Ross (CIR) dynamics affects the prices of th...
详细信息
The optimal portfolio problem for a bank account, single risky stock and a rolling horizon bond is developed. The stochastic short-term interest rate with the Cox-Ingersoll-Ross (CIR) dynamics affects the prices of the stock and rolling horizon bond. The investment objective is maximizing expected CRRA utility of terminal wealth. The problem has been solved by the stochastic dynamic programming principle and the completion of squares technique. The closed-form optimal trading strategy is obtained. A numerical example illustrating the results is presented.
We consider a capacity planning optimization problem in a general theoretical framework that extends the classical Erlang loss modeland related stochastic loss networks to support time-varying workloads. The time hori...
详细信息
ISBN:
(纸本)9781595936394
We consider a capacity planning optimization problem in a general theoretical framework that extends the classical Erlang loss modeland related stochastic loss networks to support time-varying workloads. The time horizon consists of a sequence of coarse time intervals, each of which involves a stochastic loss network under a fixed multi-class workload that can change in a general manner from one interval to the next. The optimization problem consists of determining the capacities for each time interval that maximize a utility function over the entire time horizon, finite or infinite, where rewards gained from servicing customers are offset by penalties associated with deploying capacities in an interval and with changing capacities among intervals. We derive a state-dependent optimal policy within the context of a particular limiting regime of the optimization problem, and we prove this solution to be a symptotically optimal. Then, under fairly mild conditions, we prove that a similar structural property holds for the optimal solution of the original stochastic optimization problem, and we show how the optimal capacities comprising this solution can be efficiently computed.
Platelets are short-life blood components used in hospital blood transfusion *** time for transportation,testing,and arrangement, clinically transfusable platelets have a mere three-day *** paper analyzes a periodic r...
详细信息
Platelets are short-life blood components used in hospital blood transfusion *** time for transportation,testing,and arrangement, clinically transfusable platelets have a mere three-day *** paper analyzes a periodic review inventory system for such perishable products under two replenishment *** orders are placed at the beginning of a *** the cycle, the manager has the option of placing an emergency order,characterized by an order-up-to level policy. We prove the existence and uniqueness of an optimal policy that minimizes the expected *** then derive the necessary and sufficient conditions for the policy,based on which a heuristic algorithm is developed.A numerical illustration and a sensitivity analysis are provided,along with managerial *** numerical results show that,unlike typical inventory problems,the total expected cost is sensitive to the regular order policy. It also shows that the optimal policy is sensitive to changes in the expected demand,suggesting to decision makers the significance of having an accurate demand forecast.
ABSTRACT: A modified dynamicprogramming (DP) approach that is called aggregate state dynamicprogramming (ASDP) is presented to optimally operate irrigation water delivery systems. ASDP can be applied to multiple res...
详细信息
A review is presented of the development over the years of the theory and practical use of Markov decision processes. To this purpose three periods are considered: before 1966, from 1966 till 1972, and after 1973. In ...
详细信息
It has been recognized for some time that when cost-benefit analysis is applied to irreversible environmental decisions, such as that of developing or preserving wilderness land, there can be an option value associate...
详细信息
It has been recognized for some time that when cost-benefit analysis is applied to irreversible environmental decisions, such as that of developing or preserving wilderness land, there can be an option value associated with the preservation decision, which arises when there is future uncertainty with respect to the benefits of development or preservation. In this paper the provenance of option value is examined. It is shown that an important cause is a special kind of uncertainty, viz the possibility of reversals in direction of the relative valuations of wilderness land and developed land, a property referred to as ditonicity. It is shown that the more ditonic the relative valuation process the greater the deviance between the certainty-equivalence development policy and the stochastically optimal one, and thus by implication the greater the option value. In the two cases with zero ditonicity, when relative wilderness values always increase or always decrease (even though in a stochastic fashion), there is zero option value. The model used assumes that service flows from wilderness and developed land are size-dependent, with future relative values known only in terms of a stochastic process, which can take jumps up or down of the same proportional size, at random times. Development can be partial or total and can occur in impulses at any time over an infinite time horizon. -Authors
Risk and risk preferences belong to the key determinants of investment-based technology adoption in agriculture. We develop and apply a novel approach in which an inverse second order stochastic dominance approach is ...
详细信息
Infinite horizon dynamic optimization problems with non-exponential time preferences may not only exhibit time inconsistency but may also have multiple solutions with distinct payoffs. We here show that such multiplic...
详细信息
Infinite horizon dynamic optimization problems with non-exponential time preferences may not only exhibit time inconsistency but may also have multiple solutions with distinct payoffs. We here show that such multiplicity is generic in the sense that it occurs in an open set of such decision problems, even with small state- and action-spaces. Non-exponential discounting allows for an "addictive" equilibrium alongside a "virtuous" equilibrium. We also provide a sufficient condition for uniqueness in infinitely repeated decision problems with general action spaces.
The problem of when, if ever, a stand of old-growth forest should be harvested is formulated as an optimal stopping problem, and a decision rule to maximize the expected present value of amenity services plus timber b...
详细信息
The problem of when, if ever, a stand of old-growth forest should be harvested is formulated as an optimal stopping problem, and a decision rule to maximize the expected present value of amenity services plus timber benefits is found analytically. This solution can be thought of as providing the "correct' way in which cost-benefit analysis should be carried out. It is shown that monotonicity (or lack of it) in the value of amenity services relative to timber values plays an important part in the solution. If amenity values never go down (or never go up) relative to timber values, then the certain-equivalence cost-benefit procedure provides the optimal solution, and there is no option value. It is only to the extent that the relative valuations can change direction that the certainty-equivalence procedure becomes sub-optimal and option value arises. -from Authors
暂无评论