We formulate the day-ahead bidding problem for a hydropower producer having several hydropower plants residing in a river basin. We present a novel approach inspired by dynamicprogramming with approximations in value...
详细信息
We formulate the day-ahead bidding problem for a hydropower producer having several hydropower plants residing in a river basin. We present a novel approach inspired by dynamicprogramming with approximations in value and policy space by neural networks. This allows for more accurate modeling of the problem by avoiding linear approximations of the production function and bidding. Stochastic programming is a method frequently used in literature to solve the hydropower production planning problem. Stochastic programming is used on linearized systems and under assumptions of known distributions of the involved stochastic processes. We test the proposed algorithm on a simplified system, suitable for Stochastic programming and compare the obtained policy with the results from Stochastic programming. The results show that the algorithm obtains a policy similar to that of Stochastic programming.
Resource Constraint Project Scheduling Problems with Discounted Cash Flows (RCPSPDC) focuses on maximizing the net present value by summing the discounted cash flows of project activities. An extension of this problem...
详细信息
Resource Constraint Project Scheduling Problems with Discounted Cash Flows (RCPSPDC) focuses on maximizing the net present value by summing the discounted cash flows of project activities. An extension of this problem is the Payment at Event Occurrences (PEO) scheme, where the client makes multiple payments to the contractor upon completion of predefined activities, with additional final settlement at project completion. Numerous approximation methods such as metaheuristics have been proposed to solve this NP-hard problem. However, these methods suffer from parameter control and/or the computational cost of correcting infeasible solutions. Alternatively, approximate dynamic programming (ADP) sequentially generates a schedule based on strategies computed via Monte Carlo (MC) simulations. This saves the computations required for solution corrections, but its performance is highly dependent on its strategy. In this study, we propose the hybridization of ADP with three different metaheuristics to take advantage of their combined strengths, resulting in six different models. The Estimation of Distribution Algorithm (EDA) and Ant Colony Optimization (ACO) were used to recommend policies for ADP. A Discrete cCuckoo Search (DCS) further improved the schedules generated by ADP. Our experimental analysis performed on the j30, j60, and j90 datasets of PSPLIB has shown that ADP-DCS is better than ADP alone. Implementing the EDA and ACO as prioritization strategies for Monte Carlo simulations greatly improved the solutions with high statistical significance. In addition, models with the EDA showed better performance than those with ACO and random priority, especially when the number of events increased.
In this paper, a multistage reliability-constrained stochastic planning model is established for the diamond distribution network, a new type of distribution system grid structure, with the goal of optimizing the tota...
详细信息
In this paper, a multistage reliability-constrained stochastic planning model is established for the diamond distribution network, a new type of distribution system grid structure, with the goal of optimizing the total cost of investment and operation. The uncertainty of load growth is simulated by generating multiple load growth scenarios, and the power supply loss caused by the reliability of components in the system is also considered. The multistage stochastic dynamic decision process is modeled by The Markov decision process. In order to solve the "curse of dimensionality" problem in multi-stage stochastic planning model, the approximate dynamic programming algorithm is used to solve it. The case study is based on the typical structure of the diamond distribution network and realistic data of an actual district in Shanghai, China. The planning results verify the effectiveness of the model and the solution algorithm, and demonstrate the reliability and economy of the diamond distribution network are superior to the traditional double-ring network.
The simplicity of approximate dynamic programming offers benefits for large-scale systems compared to other synthesis and control methodologies. A common technique to approximate the dynamic Program, is through the so...
详细信息
ISBN:
(纸本)9781509025923
The simplicity of approximate dynamic programming offers benefits for large-scale systems compared to other synthesis and control methodologies. A common technique to approximate the dynamic Program, is through the solution of the corresponding Linear Program. The major drawback of this approach is that the online performance is very sensitive to the choice of tuning parameters, in particular the state relevance weighting parameter. Our work aims at alleviating this sensitivity. To achieve this, we propose to find a set of approximate Q-functions, each for a different choice of the tuning parameters, and then to use the pointwise maximum of the set of Q-functions for the online policy. The pointwise maximum promises to be better than using only one of individual Q-functions for the online policy. We demonstrate that this approach immunizes against tuning errors through a stylized portfolio optimization problem.
Optimal output synchronization of multi-agent leader-follower systems is considered. The agents are assumed heterogeneous so that the dynamics may be non-identical. An optimal control protocol is designed for each age...
详细信息
ISBN:
(纸本)9781509006212
Optimal output synchronization of multi-agent leader-follower systems is considered. The agents are assumed heterogeneous so that the dynamics may be non-identical. An optimal control protocol is designed for each agent based on the leader state and the agent local state. A distributed observer is designed to provide the leader state for each agent. A model-free approximate dynamic programming algorithm is then developed to solve the optimal output synchronization problem online in real time. No knowledge of the agents' dynamics is required. The proposed approach does not require explicitly solving of the output regulator equations, though it implicitly solves them by imposing optimality. A simulation example verifies the suitability of the proposed approach.
In this paper, a novel online approximate dynamic programming(ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic(LQ) optimal control problem...
详细信息
In this paper, a novel online approximate dynamic programming(ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic(LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix,the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.
Failing to consider the many long-term uncertainties that affect the performance of a power generation portfolio can result in suboptimal generation expansion plans. Further, traditional deterministic approaches can o...
详细信息
ISBN:
(纸本)9781538655832
Failing to consider the many long-term uncertainties that affect the performance of a power generation portfolio can result in suboptimal generation expansion plans. Further, traditional deterministic approaches can omit flexible plans that are able to adapt to future events. In this paper, we explore the use of approximate dynamic programming (ADP) to create forward-looking generation expansion plans. A case study is included with three sequential decision periods;three generation technologies;and four uncertainties: demand growth, natural gas prices, renewable portfolio standards, and the adoption of customer-driven solar generation. The flexible plans found through ADP show a 3% reduction in total expected cost when compared to myopic planning heuristics while circumventing the computational burdens that accompany high-dimensional dynamic programs.
More recently,with the increasing demand of web services on the World Wide Web used in the Internet of Things (IoTs),there has been a growing interest in the study of efficient web service quality evaluation approache...
详细信息
ISBN:
(纸本)9781479980048
More recently,with the increasing demand of web services on the World Wide Web used in the Internet of Things (IoTs),there has been a growing interest in the study of efficient web service quality evaluation approaches through the use of prediction strategies to obtain accurate quality-of-service (QoS) *** unpredictable network environment imposes very challenging obstacles to web service QoS *** of the traditional web service QoS prediction approaches are implemented only using a set of static model parameters with the help of designer’s priori *** the traditional QoS prediction approaches,our algorithm in this paper is realized by incorporating approximate dynamic programming (ADP)-based online parameter tuning strategy into the QoS prediction *** online learning and optimization,the proposed approach implements the QoS prediction with automatic parameter tuning capability,and the prior knowledge or identification of the prediction model is not *** near-optimal performance of QoS prediction can therefore be *** studies are carried out to demonstrate the effectiveness of the proposed ADP-based prediction approach.
approximate dynamic programming formulation implemented with an Adaptive Critic (AC) based neural network (NN) structure has evolved as a powerful alternative technique that eliminates the need for excessive computati...
详细信息
ISBN:
(纸本)9781424477456
approximate dynamic programming formulation implemented with an Adaptive Critic (AC) based neural network (NN) structure has evolved as a powerful alternative technique that eliminates the need for excessive computations and storage requirements needed for solving the Hamilton-Jacobi-Bellman (HJB) equations. A typical AC structure consists of two interacting NNs. In this paper, a novel architecture, called the Cost Function Based Single Network Adaptive Critic (J-SNAC) is used to solve control-constrained optimal control problems. Only one network is used that captures the mapping between states and the cost function. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. A non-quadratic cost function is used that incorporates the control constraints. Necessary equations for optimal control are derived and an algorithm to solve the constrained-control problem with J-SNAC is developed. Benchmark nonlinear systems are used to illustrate the working of the proposed technique. Extensions to optimal control-constrained problems in the presence of uncertainties are also considered.
This paper analyzes quasi-random sampling techniques for approximate dynamic programming. Specifically, low-discrepancy sequences and lattice point sets are investigated and compared as efficient schemes for uniform s...
详细信息
ISBN:
(纸本)9781467361279
This paper analyzes quasi-random sampling techniques for approximate dynamic programming. Specifically, low-discrepancy sequences and lattice point sets are investigated and compared as efficient schemes for uniform sampling of the state space in high-dimensional settings. The convergence analysis of the approximate solution is provided basing on geometric properties of the two discretization methods. It is also shown that such schemes are able to take advantage of regularities of the value functions, possibly through suitable transformations of the state vector. Simulation results concerning optimal management of a water reservoirs system and inventory control are presented to show the effectiveness of the considered techniques with respect to pure-random sampling.
暂无评论