作者：
Zhao, RenboTan, Vincent Y. F.NUS
Dept Elect & Comp Engn Singapore 117583 Singapore NUS
Dept Ind Syst Engn & Management Singapore 117576 Singapore NUS
Dept Math Singapore 119076 Singapore
The multiplicativeupdate (MU) algorithm has been extensively used to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizers. ...
详细信息
The multiplicativeupdate (MU) algorithm has been extensively used to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizers. However, theoretical convergence guarantees have only been derived for a few special divergences without regularization. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizers. Our main result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the nonconvex NMF optimization problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems in machine learning and signal processing.
The multiplicativeupdate (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularization...
详细信息
ISBN:
(纸本)9781509041176
The multiplicativeupdate (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence guarantees have only been derived for a few special divergences. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizations. Our result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the NMF (optimization) problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems.
The multiplicativeupdate (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularization...
详细信息
ISBN:
(纸本)9781509041183
The multiplicativeupdate (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence guarantees have only been derived for a few special divergences. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizations. Our result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the NMF (optimization) problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems.
Transfer learning is an important technology in addressing the problem that labeled data in a target domain are difficult to collect using extensive labeled data from the source domain. Recently,an algorithm named gra...
详细信息
Transfer learning is an important technology in addressing the problem that labeled data in a target domain are difficult to collect using extensive labeled data from the source domain. Recently,an algorithm named graph co-regularized transfer learning(GTL) has shown a competitive performance in transfer learning. However, its convergence is affected by the used approximate scheme, degenerating learned results. In this paper, after analyzing convergence conditions, we propose a novel update rule using the multiplicativeupdate rule and develop a new algorithm named improved GTL(IGTL) with a strict convergence guarantee. Moreover, to prove the convergence of our method, we design a special auxiliary function whose value is intimately related to that of the objective function. Finally, the experimental results on the synthetic dataset and two real-world datasets confirm that the proposed IGTL is convergent and performs better than the compared methods.
Non-negative tensor decomposition has achieved significant success in machine learning due to its superiority in extracting the non-negative parts-based features and physically meaningful latent components from high-o...
详细信息
Non-negative tensor decomposition has achieved significant success in machine learning due to its superiority in extracting the non-negative parts-based features and physically meaningful latent components from high-order data. To improve its representation ability, hypergraph has been incorporated into the tensor decomposition model to capture the nonlinear manifold structure of data. However, previous hypergraph regularized tensor decomposition methods rely on the original data space. This may result in inaccurate manifold structure and representation performance degeneration when original data suffer from noise corruption. To solve these problems, in this paper, we propose a dynamic hypergraph regularized non-negative Tucker decomposition (DHNTD) method for multiway data analysis. Specifically, to take full advantage of the multilinear structure and nonlinear manifold of tensor data, we learn the dynamic hypergraph and non-negative low-dimensional representation in a unified framework. Moreover, we develop a multiplicativeupdate (MU) algorithm to solve our optimization problem and theoretically prove its convergence. Experimental results in clustering tasks using six image datasets demonstrate the superiority of our proposed method compared with the state-of-the-art methods.
Positive or Non-negative Matrix Factorization (NMF) is an effective technique and has been widely used for Big Data representation. It aims to find two non-negative matrices W and H whose product provides an optimal a...
详细信息
ISBN:
(纸本)9783030953881;9783030953874
Positive or Non-negative Matrix Factorization (NMF) is an effective technique and has been widely used for Big Data representation. It aims to find two non-negative matrices W and H whose product provides an optimal approximation to the original input data matrix A, such that A approximate to W * H. Although, NMF plays an important role in several applications, such as machine learning, data analysis and biomedical applications. Due to the sparsity that is caused by missing information in many high-dimension scenes (e.g., social networks, recommender systems and DNA gene expressions), the NMF method cannot mine a more accurate representation from the explicit information. Therefore, the Sparse Non-negative Matrix Factorization (SNMF) can incorporate the intrinsic geometry of the data, which is combined with implicit information. Thus, SNMF can realize a more compact representation for the sparse data. In this paper, we study the Sparse Non-negative Matrix Factorization (SNMF). We use multiplicative update algorithm (MUA) that computes the factorization by applying update on both matrices W and H. Accordingly, to address these issue, we propose a two models to implement a parallel version of SNMF on GPUs using NVIDIA CUDA framework. To optimize SNMF, we use cuSPARSE optimized library to compute the algebraic operations in MUA where sparse matrices A, W and H are stored in Compressed Sparse Row (CSR) format. At last, our contribution is validated through a series of experiments achieved on two input sets i.e. a set of randomly generated matrices and a set of benchmark matrices from real applications with different sizes and densities. We show that our algorithms allow performance improvements compared to baseline implementations. The speedup on multi-GPU platform can exceed 11x as well as the Ratio can exceed 91%.
Spectral unmixing was investigated for fast spectroscopic identification in y-emitter mixtures at low-statistics in the case of measurements performed to prevent illegal nuclear material trafficking or for in situ env...
详细信息
Spectral unmixing was investigated for fast spectroscopic identification in y-emitter mixtures at low-statistics in the case of measurements performed to prevent illegal nuclear material trafficking or for in situ environmental analysis following a radiological or nuclear accident. For that purpose, a multiplicative update algorithm based on full-spectrum analysis was tested in the case of a 3 '' x3 '' NaI(Tl) detector. Automatic decision-making was addressed using Monte Carlo calculations of decision thresholds and detection limits. The first results obtained with a portable instrument equipped with a 3 '' x3 '' NaI(Tl) detector designed for the control of food samples by non-expert users following a radiological or nuclear accident, are also presented.
This paper presents a metrological approach of spectral unmixing for automatic identification and quantitative analysis of gamma-emitting radionuclides in natural background radiation at low statistics. Based on full-...
详细信息
This paper presents a metrological approach of spectral unmixing for automatic identification and quantitative analysis of gamma-emitting radionuclides in natural background radiation at low statistics. Based on full-spectrum analysis, the proposed method relies on the maximum likelihood estimation based on Poisson statistics that accounts for the spectral signatures of the gamma-emitters to be identified and natural background. In order to obtain robust decision-making at low statistics, a sparsity constraint is implemented along with counting estimation given by spectral unmixing. In contrast with the standard approach, this technique relies on a single decision threshold applied for a likelihood ratio test. Standard deviations on estimated counting are determined using the Fisher information matrix. The robustness of decision-making and counting estimation was investigated by means of Monte Carlo calculations based on experimental spectral signatures of two types of scintillation detectors [NaI(Tl), plastic]. This study demonstrates that sparse spectral unmixing is a reliable method for gamma-spectra analysis based on low-level measurements. The sparsity constraint acts as an efficient technique for decision-making in the case of complex mixtures of gamma-emitters with significant contribution of natural background. This method also yields unbiased counting estimation related to the identified radionuclides. Reliable assessment of standard deviations are obtained and the Gaussian approximation of the coverage intervals is validated. The proposed method can be applied either by non-expert users for automatic analysis of gamma-spectra or to help experts in decision-making in the case of complex mixtures of gamma-emitters at low statistics.
The problem of dimensionality reduction is to map data from high dimensional spaces to low dimensional spaces. In the process of dimensionality reduction, the data structure, which is helpful to discover the latent se...
详细信息
The problem of dimensionality reduction is to map data from high dimensional spaces to low dimensional spaces. In the process of dimensionality reduction, the data structure, which is helpful to discover the latent semantics and simultaneously respect the intrinsic geometric structure, should be preserved. In this paper, to discover a low-dimensional embedding space with the nature of structure preservation and basis compactness, we propose a novel dimensionality reduction algorithm, called Structure Preserving Non-negative Matrix Factorization (SPNMF). In SPNMF, three kinds of constraints, namely local affinity, distant repulsion, and embedding basis redundancy elimination, are incorporated into the NMF framework. SPNMF is formulated as an optimization problem and solved by an effective iterative multiplicative update algorithm. The convergence of the proposed update solutions is proved. Extensive experiments on both synthetic data and six real world data sets demonstrate the encouraging performance of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF. Moreover, the convergence of the proposed updating rules is experimentally validated. (C) 2013 Elsevier Inc. All rights reserved.
We give sublinear-time approximation algorithms for some optimization problems arising in machine learning, such as training linear classifiers and finding minimum enclosing balls. Our algorithms can be extended to so...
详细信息
ISBN:
(纸本)9780769542447
We give sublinear-time approximation algorithms for some optimization problems arising in machine learning, such as training linear classifiers and finding minimum enclosing balls. Our algorithms can be extended to some kernelized versions of these problems, such as SVDD, hard margin SVM, and L-2-SVM, for which sublinear-time algorithms were not known before. These new algorithms use a combination of a novel sampling techniques and a new multiplicative update algorithm. We give lower bounds which show the running times of many of our algorithms to be nearly best possible in the unit-cost RAM model. We also give implementations of our algorithms in the semi-streaming setting, obtaining the first low pass polylogarithmic space and sublinear time algorithms achieving arbitrary approximation factor.
暂无评论