The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observ...
详细信息
ISBN:
(纸本)9781538679012;9781538679265
The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q-function). However, Q-learning can perform poorly when the full state observation is not available. In this paper, we formulate the POMDP estimation into a HMM estimation problem and propose a recursive algorithm to estimate both the POMDP parameter and Q-function concurrently. Also, we show that the POMDP estimation converges to a set of stationary points for the maximum likelihood estimate, and the Q-function estimation converges to a fixed point that satisfies the Bellman optimality equation weighted on the invariant distribution of the state belief determined by the HMM estimation process.
Output power of a fuel cell (FC) stack can be controlled through operating parameters (current, temperature, etc.) and is impacted by ageing and degradation. However, designing a complete FC model which includes the w...
详细信息
Output power of a fuel cell (FC) stack can be controlled through operating parameters (current, temperature, etc.) and is impacted by ageing and degradation. However, designing a complete FC model which includes the whole physical phenomena is very difficult owing to its multivariate nature. Hence, online identification of a FC model, which serves as a basis for global energy management of a fuel cell vehicle (FCV), is considerably important. In this paper, two well-known recursive algorithms are compared for online estimation of a multi-input semi-empirical FC model parameters. In this respect, firstly, a semi-empirical FC model is selected to reach a satisfactory compromise between computational time and physical meaning. Subsequently, the algorithms are explained and implemented to identify the parameters of the model. Finally, experimental results achieved by the algorithms are discussed and their robustness is investigated. The ultimate results of this experimental study indicate that the employed algorithms are highly applicable in coping with the problem of FC output power alteration, due to the uncertainties caused by degradation and operation condition variations, and these results can be utilized for designing a global energy management strategy in a FCV.
This paper presents a new method of online wind turbine performance modelling (recursive parameter estimation) that addresses the nonlinearity associated with wind turbine performance characteristics. A sliding linear...
详细信息
This paper presents a new method of online wind turbine performance modelling (recursive parameter estimation) that addresses the nonlinearity associated with wind turbine performance characteristics. A sliding linearization algorithm is implemented to track changes in the turbine operating environment. A multivariate polynomial approximation of the turbine power coefficient is developed to produce a linear process model approximating the operating wind turbine. The estimated model parameters are recursively calculated to compensate for changes in both the turbine operating environment and the condition of the wind turbine. The algorithm models both the steady-state and dynamic wind turbine performance throughout the entire operating range, producing a continuously valid turbine linearization with applications in gain scheduling and turbine performance optimization. Copyright (C) 2016 John Wiley & Sons, Ltd.
Three-phase, four-wire systems are prone to various power quality (PQ) problems such as load unbalancing, low power factor, harmonics and problem of neutral current. Modified recursive Gauss-Newton (MRGN) algorithm is...
详细信息
Three-phase, four-wire systems are prone to various power quality (PQ) problems such as load unbalancing, low power factor, harmonics and problem of neutral current. Modified recursive Gauss-Newton (MRGN) algorithm is proposed to mitigate various PQ problems in four-wire distribution systems. The recursive Gauss-Newton method is quite complex, so it is modified to obtain simple and fast MRGN control technique. The proposed algorithm is developed using MATLAB/Simulink environment and tested for mitigation of various PQ problems. Thereafter, a hardware prototype for shunt active power filter (SAPF) is developed along with various sensors, control circuits using dspace 1104 as digital signal processor. Gating circuits provide proper gating pulses to control SAPF as per the new algorithm. Results with linear and non-linear loads have been tested under a variety of loading conditions and thoroughly analysed for its performance.
Singularly perturbed linear control systems have been traditionally studied with one-vector input that controls both the slow and fast state variables. Motivated by examples of real physical systems, in this paper we ...
详细信息
Singularly perturbed linear control systems have been traditionally studied with one-vector input that controls both the slow and fast state variables. Motivated by examples of real physical systems, in this paper we extend the study to corresponding two-vector input systems. We have identified several classes of real physical singularly perturbed systems that can be more efficiently controlled using two-vector inputs. In such systems, one vector input is used to control the slow state space variables and the other vector input is used to control the fast state variables. This control structure offers more flexibility controlling given systems and appears to be more effective, We study the linear-quadratic regulator (LQR) problem for this new class of singularly perturbed systems and develop a numerical algorithm based on fixed-point iterations to obtain its solution. The latter is obtained in terms of reduced-order sub-problems which avoid possible numerical stiffness issues and are computationally advantageous for large scale systems. (C) 2017 Elsevier Ltd. All rights reserved.
A mathematical model of joint servicing of the multiservice real time traffic and elastic data traffic with batch arrival of requests was constructed and investigated. The definitions of the performance measures of jo...
详细信息
A mathematical model of joint servicing of the multiservice real time traffic and elastic data traffic with batch arrival of requests was constructed and investigated. The definitions of the performance measures of joint requests servicing were formulated, and their interrelations established. An algorithm to estimate numerically the characteristics of a model based on construction and solution of the equilibrium equations system was developed. Consideration was given to some partial cases of the model for which convenient recursive algorithms to calculate the measures of joint requests servicing were constructed. A method to estimate the desired rate of data transmission was proposed on the basis of the asymptotic properties of the model under light load. Numerical examples illustrating the distinctions of realizing the developed computer procedures were described.
We address the problem of approximating the posterior probability distribution of the fixed parameters of a state-space dynamical system using a sequential Monte Carlo method. The proposed approach relies on a nested ...
详细信息
We address the problem of approximating the posterior probability distribution of the fixed parameters of a state-space dynamical system using a sequential Monte Carlo method. The proposed approach relies on a nested structure that employs two layers of particle filters to approximate the posterior probability measure of the static parameters and the dynamic state variables of the system of interest, in a vein similar to the recent “sequential Monte Carlo square” (SMC2) algorithm. However, unlike the SMC2 scheme, the proposed technique operates in a purely recursive manner. In particular, the computational complexity of the recursive steps of the method introduced herein is constant over time. We analyse the approximation of integrals of real bounded functions with respect to the posterior distribution of the system parameters computed via the proposed scheme. As a result, we prove, under regularity assumptions, that the approximation errors vanish asymptotically in Lp (p≥1) with convergence rate proportional to 1N+1M, where N is the number of Monte Carlo samples in the parameter space and N×M is the number of samples in the state space. This result also holds for the approximation of the joint posterior distribution of the parameters and the state variables. We discuss the relationship between the SMC2 algorithm and the new recursive method and present a simple example in order to illustrate some of the theoretical findings with computer simulations.
This paper is concerned with a constrained optimization problem over a directed graph (digraph) of nodes, in which the cost function is a sum of local objectives, and each node only knows its local objective and const...
详细信息
This paper proposes an algorithm to find a common zeroing-output system (CZOS) that plays a key role in studying uniform global exponential stability for a class of switched linear time-invariant systems under arbitra...
详细信息
Wireless communication system incorporating unmanned aerial vehicles (UAVs) has gained much popularity recently, especially in video transmission application. This paper investigates the base station (BS) selection sc...
详细信息
ISBN:
(纸本)9781538663592;9781538663585
Wireless communication system incorporating unmanned aerial vehicles (UAVs) has gained much popularity recently, especially in video transmission application. This paper investigates the base station (BS) selection scheme for cellular-connected UAVs that possess the function of video collection and streaming to BS for online decision in remote processing center. We aim to minimize the expected access latency while the throughput requirement is satisfied. To this end, a sequential BS selection scheme is proposed by designing an optimal transmission rate threshold for each candidate BS. Due to the mission-driven nature, the access rate varies as the UAV moves, so an effective average transmission rate rather than instantaneous transmission rate is considered. A recursive algorithm is proposed to obtain the rate thresholds which can be used to guide whether UAV should stop or continue measuring the links of the remaining candidate BSs. The proof of the optimality of this algorithm is given. Simulation results validate the effectiveness of the proposed scheme on access latency performance compared with conventional throughput-oriented scheme.
暂无评论