The expectation-maximization (EM) algorithm is a very popular optimization tool for mixture problems and in particular for model-based clustering problems. However, while the algorithm is convenient to implement and n...
详细信息
The expectation-maximization (EM) algorithm is a very popular optimization tool for mixture problems and in particular for model-based clustering problems. However, while the algorithm is convenient to implement and numerically very stable, it only produces local solutions. Thus, it may not achieve the globally optimal solution in problems that have a large number of local optima. This paper introduces several new algorithms designed to produce global solutions in model-based clustering. The building blocks for these algorithms are methods from the operations research literature, namely the Cross-Entropy (CE) method and Model Reference Adaptive Search (MRAS). One problem with applying these methods directly is the efficient simulation of positive definite covariance matrices. We propose several new solutions to this problem. One solution is to apply the principles of expectation-maximization updating, which leads to two new algorithms, CE-EM and MRAS-EM. We also propose two additional algorithms, CE-CD and MRAS-CD, which rely on the Cholesky decomposition. We conduct numerical experiments of varying complexity to evaluate the effectiveness of the proposed algorithms in comparison to classical EM. We find that although a single run of the new algorithms is slower than a single run of EM, all have the potential for producing significantly better solutions. We also find that although repeat application of EM may achieve similar results, our algorithms provide automated, data-driven decision rules which may significantly reduce the burden of searching for the global optimum. (C) 2009 Elsevier B.V. All rights reserved.
There is a need for efficient methods for estimating trends in spatio-temporal Earth Observation data. A suitable model for such data is a space-varying regression model, where the regression coefficients for the spat...
详细信息
There is a need for efficient methods for estimating trends in spatio-temporal Earth Observation data. A suitable model for such data is a space-varying regression model, where the regression coefficients for the spatial locations are dependent. A second order intrinsic Gaussian Markov Random Field prior is used to specify the spatial covariance structure. Model parameters are estimated using the expectation Maximisation (EM) algorithm, which allows for feasible computation times for relatively large data sets. Results are illustrated with simulated data sets and real vegetation data from the Sahel area in northern Africa. The results indicate a substantial gain in accuracy compared with methods based on independent ordinary least squares regressions for the individual pixels in the data set. Use of the EM algorithm also gives a substantial performance gain over Markov Chain Monte Carlo-based estimation approaches. (c) 2008 Elsevier B.V. All rights reserved.
We present an approach for exact maximum likelihood estimation of parameters from univariate and multivariate autoregressive fractionally integrated moving average models with Gaussian errors using the expectation Max...
详细信息
We present an approach for exact maximum likelihood estimation of parameters from univariate and multivariate autoregressive fractionally integrated moving average models with Gaussian errors using the expectationmaximization (EM) algorithm. The method takes advantage of the relation between the VARFIMA(0, d, 0) process and the corresponding VARFIMA(p, d, q) process in the computation of the likelihood. (C) 2009 Elsevier B.V. All rights reserved.
Study of dynamic processes in many areas of science has led to the appearance of functional data sets. It is often the case that individual trajectories vary both in the amplitude space and in the time space. We devel...
详细信息
Study of dynamic processes in many areas of science has led to the appearance of functional data sets. It is often the case that individual trajectories vary both in the amplitude space and in the time space. We develop a coherent clustering procedure that allows for temporal aligning. Under this framework, closed form solutions of an EM type learning algorithm are derived. The method can be applied to all types of curve data but is particularly useful when phase variation is present. We demonstrate the method by both simulation studies and an application to human growth curves. Published by Elsevier B.V.
We examine the continuous time analogue of the work of [Shumway, R.H., Stoffer, D.S., 1982. An approach to time series smoothing and forecasting using EM algorithm. J. Time Ser. 3, 253-264] for state space models when...
详细信息
We examine the continuous time analogue of the work of [Shumway, R.H., Stoffer, D.S., 1982. An approach to time series smoothing and forecasting using EM algorithm. J. Time Ser. 3, 253-264] for state space models when the noise in the observation process is a fractional Brownian motion. We Study the estimation problem for the parameter of the system process. (c) 2008 Elsevier B.V. All rights reserved.
作者：
Chen, ZheVijayan, SujithBarbieri, RiccardoWilson, Matthew A.Brown, Emery N.Harvard Univ
Massachusetts Gen Hosp Sch Med Neurosci Stat Res LabDept Anesthesia & Crit Care Boston MA 02114 USA MIT
Harvard Mit Div Hlth Sci & Technol Cambridge MA 02139 USA Harvard Univ
Program Neurosci Cambridge MA 02139 USA MIT
Dept Biol Cambridge MA 02139 USA MIT
Picower Inst Learning & Memory RIKEN MIT Neurosci Res Ctr Dept Brain & Cognit Sci Cambridge MA 02139 USA
UP and DOWN states, the periodic fluctuations between increased and decreased spiking activity of a neuronal population, are a fundamental feature of cortical circuits. Understanding UP-DOWN state dynamics is importan...
详细信息
UP and DOWN states, the periodic fluctuations between increased and decreased spiking activity of a neuronal population, are a fundamental feature of cortical circuits. Understanding UP-DOWN state dynamics is important for understanding how these circuits represent and transmit information in the brain. To date, limited work has been done on characterizing the stochastic properties of UP-DOWN state dynamics. We present a set of Markov and semi-Markov discrete-and continuous-time probability models for estimating UP and DOWN states from multiunit neural spiking activity. We model multiunit neural spiking activity as a stochastic point process, modulated by the hidden (UP and DOWN) states and the ensemble spiking history. We estimate jointly the hidden states and the model parameters by maximum likelihood using an expectation-maximization (EM) algorithm and a Monte Carlo EM algorithm that uses reversible-jump Markov chain Monte Carlo sampling in the E-step. We apply our models and algorithms in the analysis of both simulated multiunit spiking activity and actual multiunit spiking activity recorded from primary somatosensory cortex in a behaving rat during slow-wave sleep. Our approach provides a statistical characterization of UP-DOWN state dynamics that can serve as a basis for verifying and refining mechanistic descriptions of this process.
Message passing algorithms have had dramatic impacts on important problems in signal processing, learning theory, communication theory, and information theory through their computational efficiency. expectation-maximi...
详细信息
ISBN:
(纸本)0780394038
Message passing algorithms have had dramatic impacts on important problems in signal processing, learning theory, communication theory, and information theory through their computational efficiency. expectation-maximization algorithms have had dramatic impacts on problems in estimation and detection theory, but their computational efficiency often limits their applicability. Given a bipartite graphical model for the data, if a set of hidden independent random variables can be associated with the edges, then a resulting expectation-maximization algorithm is message passing on this graph. The algorithms are computationally efficient in the same sense as other message passing algorithms. One example of such algorithms is the standard expectation-maximization algorithm for emission tomography. Another example for a signal in Gaussian noise yields a statistical interpretation to efficient algorithms for sparse linear inverse problems.
This paper addresses the problem of multi-pitch estimation, which consists in estimating the fundamental frequencies of multiple harmonic sources, with possibly overlapping partials, from their mixture. The proposed a...
详细信息
This paper addresses the problem of multi-pitch estimation, which consists in estimating the fundamental frequencies of multiple harmonic sources, with possibly overlapping partials, from their mixture. The proposed approach is based on the expectation-maximization algorithm, which aims at maximizing the likelihood of the observed spectrum, by performing successive single-pitch and spectral envelope estimations. This algorithm is illustrated in the context of musical chord identification.
We consider inference in a general data-driven object-based model of multichannel audio data, assumed generated as a possibly under-determined convolutive mixture of source signals. Each source is given a model inspir...
详细信息
We consider inference in a general data-driven object-based model of multichannel audio data, assumed generated as a possibly under-determined convolutive mixture of source signals. Each source is given a model inspired from nonnegative matrix factorization (NMF) with the Itakura-Saito divergence, which underlies a statistical model of superimposed Gaussian components. We address estimation of the mixing and source parameters using two methods. The first one consists of maximizing the exact joint likelihood of the multichannel data using an expectation-maximization algorithm. The second method consists of maximizing the sum of individual likelihoods of all channels using a multiplicative update algorithm inspired from NMF methodology. Our decomposition algorithms were applied to stereo music and assessed in terms of blind source separation performance.
In this paper we present a supervised method of image segmentation based on the statistic approach expressed in an hybrid space constituted by the three relevant chromatic level deduced by histogram analysis approach,...
详细信息
In this paper we present a supervised method of image segmentation based on the statistic approach expressed in an hybrid space constituted by the three relevant chromatic level deduced by histogram analysis approach, this technique may the possibility of adapting the treatments to the local context of image with a little priori knowledge. This method has been applied on colour images issued from a soccer video of football sports. The obtained results show how his method reconstructs faithfully the size of different region while discriminating textures areas. We next study the influence of statistical parameters and chromatic level on these results.
暂无评论