In this paper we address the problem of sparse representation (SR) within a Bayesian framework. We assume that the observations are generated from a Bernoulli-Gaussian process and consider the corresponding Bayesian i...
详细信息
ISBN:
(纸本)9781424442959
In this paper we address the problem of sparse representation (SR) within a Bayesian framework. We assume that the observations are generated from a Bernoulli-Gaussian process and consider the corresponding Bayesian inference problem. Tractable solutions are then proposed based on the "mean-field" approximation and the variational Bayes EM algorithm. The resulting SR algorithms are shown to have a tractable complexity and very good performance over a wide range of sparsity levels. In particular, they significantly improve the critical sparsity upon state-of-the-art SR algorithms.
Bayesian Approximate Message Passing (BAMP) provides excellent recovery performance in Compressed Sensing (CS), but one seemingly needs to know the pdf of the signal prior. If the shape of the pdf is known but not its...
详细信息
Bayesian Approximate Message Passing (BAMP) provides excellent recovery performance in Compressed Sensing (CS), but one seemingly needs to know the pdf of the signal prior. If the shape of the pdf is known but not its parameters, we show how they can be estimated with very low complexity during the BAMP iterations by the well-known Method of Moments (MoM). We compare the new approach with an established scheme from the literature that is based on the expectation Maximization (EM) algorithm. By simulations we show that the MoM-based BAMP scheme works at least as good as the EM-based approach and with much lower complexity.
Machine learning has reached a point where many probabilistic methods can be understood as variations, extensions and combinations of a much smaller set of abstract themes, e.g., as different instances of the EM algor...
详细信息
ISBN:
(纸本)0262025507
Machine learning has reached a point where many probabilistic methods can be understood as variations, extensions and combinations of a much smaller set of abstract themes, e.g., as different instances of the EM algorithm. This enables the systematic derivation of algorithms customized for different models. Here, we describe the AUTOBAYES system which takes a high-level statistical model specification, uses powerful symbolic techniques based on schema-based program synthesis and computer algebra to derive an efficient specialized algorithm for learning that model, and generates executable code implementing that algorithm. This capability is far beyond that of code collections such as Matlab toolboxes or even tools for model-independent optimization such as BUGS for Gibbs sampling: complex new algorithms can be generated without new programming, algorithms can be highly specialized and tightly crafted for the exact structure of the model and data, and efficient and commented code can be generated for different languages or systems. We present automatically-derived algorithms ranging from closed-form solutions of Bayesian textbook problems to recently-proposed EM algorithms for clustering, regression, and a multinomial form of PCA.
In this paper, we study the time of arrival (TOA)-based distributed passive localization in asynchronous wireless network. Performing synchronization between receivers before target localization is possible but costs ...
详细信息
ISBN:
(纸本)9781479959532
In this paper, we study the time of arrival (TOA)-based distributed passive localization in asynchronous wireless network. Performing synchronization between receivers before target localization is possible but costs extra energy and band-width. To this end, We propose an expectation maximization (EM) algorithm to locate the passive target in the presence of receivers' clock offsets. To improve the robustness of the proposed algorithm, we employ the average consensus scheme to obtain the location of target at each receiver in a distributed way. A quadratic polynomial approximation is proposed to reduce the communication overhead and computational complexity. To evaluate the performance of the proposed algorithm, the Cramer-Rao bound (CRB) of the target's position estimation is derived. Simulation results show that the proposed distributed EM algorithm performs close to the centralized counterpart. It outperforms the conventional two step estimation method and the one based on time difference of arrival. Moreover, the proposed algorithm can attain the derived CRB, which demonstrates the effectiveness of the algorithm.
We present new training methods that aim to mitigate local optima and slow convergence in unsupervised training by using additional imperfect objectives. In its simplest form, lateen EM alternates between the two obje...
详细信息
ISBN:
(纸本)9781937284114
We present new training methods that aim to mitigate local optima and slow convergence in unsupervised training by using additional imperfect objectives. In its simplest form, lateen EM alternates between the two objectives of ordinary "soft" and "hard" expectation maximization (EM) algorithms. Switching objectives when stuck can help escape local optima. We find that applying a single such alternation already yields state-of-the-art results for English dependency grammar induction. More elaborate lateen strategies track both objectives, with each validating the moves proposed by the other. Disagreements can signal earlier opportunities to switch or terminate, saving iterations. De-emphasizing fixed points in these ways eliminates some guesswork from tuning EM. An evaluation against a suite of unsupervised dependency parsing tasks, for a variety of languages, showed that lateen strategies significantly speed up training of both EM algorithms, and improve accuracy for hard EM.
An important challenge in speech processing involves extracting non-linguistic information from a fundamental frequency (F_0) contour of speech. We propose a fast algorithm for estimating the model parameters of the F...
详细信息
ISBN:
(纸本)9781509041183
An important challenge in speech processing involves extracting non-linguistic information from a fundamental frequency (F_0) contour of speech. We propose a fast algorithm for estimating the model parameters of the Fujisaki model, namely, the timings and magnitudes of the phrase and accent commands. Although a powerful parameter estimation framework based on a stochastic counterpart of the Fujisaki model has recently been proposed, it still had room for improvement in terms of both computational efficiency and parameter estimation accuracy. This paper describes our two contributions. First, we propose a hard expectation-maximization (EM) algorithm for parameter inference where the E step of the conventional EM algorithm is replaced with a point estimation procedure to accelerate the estimation process. Second, to improve the parameter estimation accuracy, we add a generative process of a spectral feature sequence to the generative model. This makes it possible to use linguistic or phonological information as an additional clue to estimate the timings of the accent commands. The experiments confirmed that the present algorithm was approximately 16 times faster and estimated parameters about 3% more accurately than the conventional algorithm.
Point processes have many engineering applications and perhaps the most used dynamic system identification model is the Hawkes model. We propose a new approach to maximum likelihood estimation of Hawkes point process ...
详细信息
ISBN:
(纸本)9781479978878
Point processes have many engineering applications and perhaps the most used dynamic system identification model is the Hawkes model. We propose a new approach to maximum likelihood estimation of Hawkes point process models. Although an EM algorithm has previously been given, it turns out to be unreliable in practice. We show that this is because it does not guarantee the stability condition required for the Hawkes process. Our new approach guarantees stability at each iteration. We illustrate with simulations and application to financial data.
Dimensionality reduction techniques such as principal component analysis and factor analysis are used to discover a linear mapping between high dimensional data samples and points in a lower dimensional subspace. In [...
详细信息
ISBN:
(纸本)0262025507
Dimensionality reduction techniques such as principal component analysis and factor analysis are used to discover a linear mapping between high dimensional data samples and points in a lower dimensional subspace. In [6], Jojic and Frey introduced mixture of transformation-invariant component analyzers (MTCA) that can account for global transformations such as translations and rotations, perform clustering and learn local appearance deformations by dimensionality reduction. However, due to enormous computational requirements of the EM algorithm for learning the model, O(N~2) where N is the dimensionality of a data sample, MTCA was not practical for most applications. In this paper, we demonstrate how fast Fourier transforms can reduce the computation to the order of N log N. With this speedup, we show the effectiveness of MTCA in various applications - tracking, video textures, clustering video sequences, object recognition, and object detection in images.
Model-based object clustering is a very challenging unsupervised learning problem in computer vision, which involves both high dimensionality and hidden variables inference issues. In this paper, we will study object ...
详细信息
ISBN:
(纸本)9781467381741
Model-based object clustering is a very challenging unsupervised learning problem in computer vision, which involves both high dimensionality and hidden variables inference issues. In this paper, we will study object pattern clustering problem by using the Active basis model, which is a sparse representation model for object patterns. We fit a mixture of active basis models, which leads to an object clustering result, as well as the inference of all the hidden variables in the object clustering problem. This strategy not only gives us a way to represent each object by a sparse model, but also an elegant solution to the clustering problem with hidden variables, such as unknown locations, scales, and orientations of the objects appearing in the images. The experiment conducted on a small clustering dataset shows that learning a mixture of active basis models by EM-like algorithm for object clustering is very promising.
In this paper we compare several channel estimation algorithms in the case of LTE (Long Term Evolution) System when applied in a high mobility environment. In particular, we propose a novel algorithm, based on the obs...
详细信息
ISBN:
(纸本)9781467318808
In this paper we compare several channel estimation algorithms in the case of LTE (Long Term Evolution) System when applied in a high mobility environment. In particular, we propose a novel algorithm, based on the observation of OFDM block energy, to perform a semi-blind tracking of the channel between pilot blocks. This algorithm is compared to more traditional approaches based on ID estimation and interpolation. In addition, an expectation Maximization (EM) algorithm is used to improve the overall channel estimation.
暂无评论