检索结果-内蒙古大学图书馆

Approximate message passing with restricted Boltzmann machine priors

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2016年第7期2016卷

作者： Tramel, Eric W. Dremeau, Angelique Krzakala, Florent Sorbonne Univ F-75005 Paris France Univ Paris 06 F-75005 Paris France Ecole Normale Super Lab Phys Stat UMR CNRS 8550 Rue Lhomond F-75005 Paris France ENSTA Bretagne Lab STICC UMR 6285 F-29806 Brest France

Approximate message passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problems. The AMP framework provides modularity in the choice of signal prior;here we propose a hierarchical form of the Gauss-Bernoulli prior which utilizes a restricted Boltzmann machine (RBM) trained on the signal support to push reconstruction performance beyond that of simple i.i.d. priors for signals whose support can be well represented by a trained binary RBM. We present and analyze two methods of RBM factorization and demonstrate how these affect signal reconstruction performance within our proposed algorithm. Finally, using the MNIST handwritten digit dataset, we show experimentally that using an RBM allows AMP to approach oracle-support performance.

关键词： message-passing algorithms random graphs networks statistical inference

来源：评论

学校读者我要写书评

暂无评论

Aligning random graphs with a sub-tree similarity message-passing algorithm

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2022年第6期2022卷 063401-063401页

作者： Piccioli, Giovanni Semerjian, Guilhem Sicuro, Gabriele Zdeborova, Lenka Ecole Polytech Fed Lausanne SPOC Lab CH-1015 Lausanne Switzerland Sorbonne Univ Univ PSL Univ Paris Cite CNRSLab PhysEcole Normale Super F-75005 Paris France Kings Coll London London WC2R 2LS England

The problem of aligning Erdos-Renyi random graphs is a noisy, average-case version of the graph isomorphism problem, in which a pair of correlated random graphs is observed through a random permutation of their vertices. We study a polynomial time message-passing algorithm devised to solve the inference problem of partially recovering the hidden permutation, in the sparse regime with constant average degrees. We perform extensive numerical simulations to determine the range of parameters in which this algorithm achieves partial recovery. We also introduce a generalized ensemble of correlated random graphs with prescribed degree distributions, and extend the algorithm to this case.

关键词： analysis of algorithms message-passing algorithms random graphs networks statistical inference

来源：评论

学校读者我要写书评

暂无评论

Matrix completion based on Gaussian parameterized belief propagation

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2021年第9期2021卷

作者： Okajima, Koki Kabashima, Yoshiyuki Univ Tokyo Grad Sch Sci Bunkyo Ku Tokyo 1130033 Japan

We develop a message-passing algorithm for noisy matrix completion problems based on matrix factorization. The algorithm is derived by approximating message distributions of belief propagation with Gaussian distributions that share the same first and second moments. We also derive a memory-friendly version of the proposed algorithm by applying a perturbation treatment commonly used in the literature of approximate message passing. In addition, a damping technique, which is demonstrated to be crucial for optimal performance, is introduced without computational strain, and the relationship to the message-passing version of alternating least squares, a method reported to be optimal in certain settings, is discussed. Experiments on synthetic datasets show that while the proposed algorithm quantitatively exhibits almost the same performance under settings where the earlier algorithm is optimal, it is advantageous when the observed datasets are corrupted by non-Gaussian noise. Experiments on real-world datasets also emphasize the performance differences between the two algorithms.

关键词： message-passing algorithms

来源：评论

学校读者我要写书评

暂无评论

The cavity approach to parallel dynamics of Ising spins on a graph

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2009年第8期2009卷 P08009-P08009页

作者： Neri, I. Bolle, D. Katholieke Univ Leuven Inst Theoret Fys B-3001 Leuven Belgium

We use the cavity method to study the parallel dynamics of disordered Ising models on a graph. In particular, we derive a set of recursive equations in single-site probabilities of paths propagating along the edges of the graph. These equations are analogous to the cavity equations for equilibrium models and are exact on a tree. On graphs with exclusively directed edges we find an exact expression for the stationary distribution. We present the phase diagrams for an Ising model on an asymmetric Bethe lattice and for a neural network with Hebbian interactions on an asymmetric scale-free graph. For graphs with a nonzero fraction of symmetric edges the equations can be solved for a finite number of time steps. Theoretical predictions are confirmed by simulations. Using a heuristic method the cavity equations are extended to a set of equations that determine the marginals of the stationary distribution of Ising models on graphs with a nonzero fraction of symmetric edges. The results from this method are discussed and compared with simulations.

关键词： cavity and replica method disordered systems (theory) message-passing algorithms

来源：评论

学校读者我要写书评

暂无评论

Mismatching as a tool to enhance algorithmic performances of Monte Carlo methods for the planted clique model

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2021年第11期2021卷

作者： Angelini, Maria Chiara Fachin, Paolo de Feo, Simone Sapienza Univ Roma Dipartimento Fis Ple A Moro 5 I-00185 Rome Italy Ist Nazl Fis Nucl Sez Roma1 Ple A Moro 5 I-00185 Rome Italy

Over-parametrization was a crucial ingredient for recent developments in inference and machine-learning fields. However a good theory explaining this success is still lacking. In this paper we study a very simple case of mismatched over-parametrized algorithm applied to one of the most studied inference problem: the planted clique problem. We analyze a Monte Carlo (MC) algorithm in the same class of the famous Jerrum algorithm. We show how this MC algorithm is in general suboptimal for the recovery of the planted clique. We show however how to enhance its performances by adding a (mismatched) parameter: the temperature;we numerically find that this over-parametrized version of the algorithm can reach the supposed algorithmic threshold for the planted clique problem.

关键词： inference of graphical models classical Monte Carlo simulations statistical inference message-passing algorithms

来源：评论

学校读者我要写书评

暂无评论

Gauges, loops, and polynomials for partition functions of graphical models

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2020年第12期2020卷

作者： Chertkov, Michael Chernyak, Vladimir Maximov, Yury Univ Arizona Program Appl Math Tucson AZ 85721 USA Skolkovo Inst Technol Moscow Russia Wayne State Univ Dept Chem Detroit MI 48202 USA Los Alamos Natl Lab T Div Los Alamos NM USA

Graphical models represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function, is the main inference challenge relevant to multiple statistical and optimization applications. The problem is #P-hard that is of an exponential complexity with respect to the number of variables. In this manuscript, aimed at approximating the partition function, we consider multi-graph models where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multi-graph. We suggest a new methodology for analysis and computations that combines the Gauge function technique from Chertkov and Chernyak (2006 Phys. Rev. E 73 065102;2006 J. Stat. Mech. 2006 P06009) with the technique developed in Anari and Oveis Gharan 2017 arXiv:;Gurvits 2011 arXiv:;Straszak and Vishnoi 2017 55th Annual Allerton Conf. on Communication, Control, and Computing, based on the recent progress in the field of real stable polynomials. We show that the Gauge function, representing a single-out term in a finite sum expression for the partition function which achieves extremum at the so-called belief-propagation gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, Gauge function can be used to recover the partition function through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists of the application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination/contraction of edges resulting in multi-graph models on decreasing in size (number of edges) graphs with the same partition function as in the original multi-graph model. Even though the complexity of computing factors in the sequence of the derived multi-graph models and respective Gauge functions gro

关键词： inference of graphical models message-passing algorithms statistical inference machine learning

来源：评论

学校读者我要写书评

暂无评论

Theory and learning protocols for the material tempotron model

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2013年第12期2013卷

作者： Baldassi, Carlo Braunstein, Alfredo Zecchina, Riccardo Politecn Torino DISAT I-10129 Turin Italy Politecn Torino Ctr Comp Sci I-10129 Turin Italy Human Genet Fdn I-10126 Turin Italy Coll Carlo Alberto I-10024 Turin Italy

Neural networks are able to extract information from the timing of spikes. Here we provide new results on the behavior of the simplest neuronal model which is able to decode information embedded in temporal spike patterns, the so-called tempotron. Using statistical physics techniques we compute the capacity for the case of sparse, time-discretized input, and 'material' discrete synapses, showing that the device saturates the information theoretic bounds with a statistics of output spikes that is consistent with the statistics of the inputs. We also derive two simple and highly efficient learning algorithms which are able to learn a number of associations which are close to the theoretical limit. The simplest versions of these algorithms correspond to distributed online protocols of interest for neuromorphic devices, and can be adapted to address the more biologically relevant continuous-time version of the classification problem, hopefully allowing the understanding of some aspects of synaptic plasticity.

关键词： message-passing algorithms neural code computational neuroscience learning theory

来源：评论

学校读者我要写书评

暂无评论

Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2018年第3期2018卷 033404-033404页

作者： Sakata, Ayaka Xu, Yingying Inst Stat Math Midori Cho Tachikawa Tokyo 1908562 Japan Grad Univ Adv Sci SOKENDAI Hayama Cho Hayama Kanagawa 2400193 Japan Aalto Univ Sch Sci Dept Comp Sci POB 15400 FI-00076 Aalto Finland

We analyse a linear regression problem with nonconvex regularization called smoothly clipped absolute deviation (SCAD) under an overcomplete Gaussian basis for Gaussian random data. We propose an approximate message passing (AMP) algorithm considering nonconvex regularization, namely SCAD-AMP, and analytically show that the stability condition corresponds to the de Almeida-Thouless condition in spin glass literature. Through asymptotic analysis, we show the correspondence between the density evolution of SCAD-AMP and the replica symmetric (RS) solution. Numerical experiments confirm that for a su. ciently large system size, SCAD-AMP achieves the optimal performance predicted by the replica method. Through replica analysis, a phase transition between replica symmetric and replica symmetry breaking (RSB) region is found in the parameter space of SCAD. The appearance of the RS region for a nonconvex penalty is a significant advantage that indicates the region of smooth landscape of the optimization problem. Furthermore, we analytically show that the statistical representation performance of the SCAD penalty is better than that of l(1)-based methods, and the minimum representation error under RS assumption is obtained at the edge of the RS/RSB phase. The correspondence between the convergence of the existing coordinate descent algorithm and RS/RSB transition is also indicated.

关键词： message-passing algorithms analysis of algorithms learning theory

来源：评论

学校读者我要写书评

暂无评论

Matrix inference and estimation in multi-layer models*

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2021年第12期2021卷

作者： Pandit, Parthe Sahraee-Ardakan, Mojtaba Rangan, Sundeep Schniter, Philip Fletcher, Alyson K. UC Los Angeles Dept ECE Los Angeles CA 90095 USA UC Los Angeles Dept Stat Los Angeles CA 90095 USA NYU Dept ECE New York NY USA Ohio State Univ Dept ECE Columbus OH 43210 USA

We consider the problem of estimating the input and hidden variables of a stochastic multi-layer neural network (NN) from an observation of the output. The hidden variables in each layer are represented as matrices with statistical interactions along both rows as well as columns. This problem applies to matrix imputation, signal recovery via deep generative prior models, multi-task and mixed regression, and learning certain classes of two-layer NNs. We extend a recently-developed algorithm-multi-layer vector approximate message passing, for this matrix-valued inference problem. It is shown that the performance of the proposed multi-layer matrix vector approximate message passing algorithm can be exactly predicted in a certain random large-system limit, where the dimensions N x d of the unknown quantities grow as N -> infinity with d fixed. In the two-layer neural-network learning problem, this scaling corresponds to the case where the number of input features as well as training samples grow to infinity but the number of hidden nodes stays fixed. The analysis enables a precise prediction of the parameter and test error of the learning.

关键词： inference of graphical models machine learning message-passing algorithms statistical inference

来源：评论

学校读者我要写书评

暂无评论

On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2009年第9期2009卷 P09001-P09001页

作者： Ricci-Tersenghi, Federico Semerjian, Guilhem Univ Roma La Sapienza Dipartimento Fis I-00185 Rome Italy Univ Paris 06 LPTENS CNRS UMR 8549 F-75231 Paris 05 France Univ Paris 06 ENS Associee F-75231 Paris 05 France

We introduce a version of the cavity method for diluted mean field spin models that allows the computation of thermodynamic quantities similar to the Franz-Parisi quenched potential in sparse random graph models. This method is developed in the particular case of partially decimated random constraint satisfaction problems. This allows us to develop a theoretical understanding of a class of algorithms for solving constraint satisfaction problems, in which elementary degrees of freedom are sequentially assigned according to the results of a message passing procedure (belief propagation). We confront this theoretical analysis with the results of extensive numerical simulations.

关键词： cavity and replica method analysis of algorithms message-passing algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：