We analyze the dynamics of a random sequential messagepassing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the...
详细信息
We analyze the dynamics of a random sequential messagepassing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica-symmetric ansatz for the static probabilistic model.
Semi-supervised learning (SSL) is a machine learning methodology that leverages unlabeled data in conjunction with a limited amount of labeled data. Although SSL has been applied in various applications and its effect...
详细信息
Semi-supervised learning (SSL) is a machine learning methodology that leverages unlabeled data in conjunction with a limited amount of labeled data. Although SSL has been applied in various applications and its effectiveness has been empirically demonstrated, it is still not fully understood when and why SSL performs well. Several existing theoretical studies have attempted to address this issue by modeling classification problems using the so-called Gaussian mixture model (GMM). These studies provide notable and insightful interpretations. However, their analyses are focused on specific purposes, and a thorough investigation of the properties of GMM in the context of SSL has been lacking. In this paper, we conduct a detailed analysis of the properties of the high-dimensional GMM for binary classification in the SSL setting. To this end, we employ the approximate message-passing and state evolution methods, which are widely used in high-dimensional settings and originate from statistical mechanics. We deal with two estimation approaches: the Bayesian one and the & ell;(2)-regularized maximum likelihood estimation (RMLE). We conduct a comprehensive comparison of these two approaches, examining aspects such as the global phase diagram, estimation error for the parameters, and prediction error for the labels. A specific comparison is made between the Bayes-optimal (BO) estimator and RMLE, as the BO setting provides the optimal estimation performance and is ideal as a benchmark. Our analysis shows that with appropriate regularizations, RMLE can achieve a near-optimal performance in terms of both the estimation error and prediction error, especially when there is a large amount of unlabeled data. These results demonstrate that the & ell;(2) regularization term plays an effective role in estimation and prediction in SSL approaches.
Industrial Internet of Things networks require large-volume data delivery across interdependent mission-critical components. This imposes stringent ultrareliable low-latency communication requirements. In this regard,...
详细信息
Industrial Internet of Things networks require large-volume data delivery across interdependent mission-critical components. This imposes stringent ultrareliable low-latency communication requirements. In this regard, cell-free network architecture has risen as a compelling solution to shorten distances between devices and access points (APs). In cell-free networks, APs simultaneously serve devices with shared time-frequency resources, utilizing channel state information acquired via pilot signals from devices. However, a limited number of orthogonal pilot sequences entails the pilot reuse across multiple links. This results in the interference among pilot signals, which, in turn, degrades the overall link utilities. A skillful pilot assignment (PA) mitigates such interference, while the combinatorial nature of handling pilot-sharing groups limits the development of an efficient protocol. This work develops a survey propagation-inspired distributed PA framework, originating from statistical physics to address the equilibrium among particle interactions, which successfully interprets the consensus among pilot-sharing groups in the PA task. This facilitates distributed and efficient addressing of complex solution spaces, leading to computation-efficient solutions.
We begin with an exact expression for the entropy of a system of hard spheres within the Hamming space. This entropy relies on probability marginals, which are determined by an extended set of belief propagation (BP) ...
详细信息
We begin with an exact expression for the entropy of a system of hard spheres within the Hamming space. This entropy relies on probability marginals, which are determined by an extended set of belief propagation (BP) equations. The BP probability marginals are functions of auxiliary variables that are introduced to model the effects of loopy interactions on a tree-structured interaction graph. We explore various reasonable and approximate probability distributions, ensuring that they align with the exact solutions of the BP equations. Our approach is based on an ansatz for (in)homogeneous cavity marginals respecting the permutation symmetry of the problem. Through a thorough analysis, we aim to minimize errors in the BP equations. Our findings support the conjecture that the maximum packing density asymptotically conforms to the lower bound proposed by Gilbert and Varshamov. The presented formulation of the problem is useful to obtain upper bounds for the packing entropy within a class of BP probability marginals. Moreover, it enables different approximations in both the structure of BP messages and the role of auxiliary variables.
We study two free energy approximations (Bethe and plaquette-CVM) for the Random Field Ising Model in two dimensions. We compare results obtained by these two methods in single instances of the model on the square gri...
详细信息
We study two free energy approximations (Bethe and plaquette-CVM) for the Random Field Ising Model in two dimensions. We compare results obtained by these two methods in single instances of the model on the square grid, showing the difficulties arising in defining a robust critical line. We also attempt average case calculations using a replica-symmetric ansatz, and compare the results with single instances. Both, Bethe and plaquette-CVM approximations present a similar panorama in the phase space, predicting long range order at low temperatures and fields. We show that plaquette-CVM is more precise, in the sense that predicts a lower critical line (the truth being no line at all). Furthermore, we give some insight on the non-trivial structure of the fixed points of different messagepassingalgorithms. A study of the Monte Carlo dynamics for an arbitrary sample shows that GBP states are very well correlated with states that are attractors of the stochastic dynamics.
The expectation consistent (EC) approximation framework is a state-of-the-art approach for solving (generalized) linear inverse problems with high-dimensional random forward operators and i.i.d. signal priors. In imag...
详细信息
ISBN:
(纸本)9798350325744
The expectation consistent (EC) approximation framework is a state-of-the-art approach for solving (generalized) linear inverse problems with high-dimensional random forward operators and i.i.d. signal priors. In image inverse problems, however, both the forward operator and image pixels are structured, which plagues traditional EC implementations. In this work, we propose a novel incarnation of EC that exploits deep neural networks to handle structured operators and signals. For phase-retrieval, we propose a simplified variant called "deepECpr" that reduces to iterative denoising. In experiments recovering natural images from phaseless, shot-noise corrupted, coded-diffraction-pattern measurements, we observe accuracy surpassing the state-of-the-art prDeep (Metzler et al., 2018) and Diffusion Posterior Sampling (Chung et al., 2023) approaches with two-orders-of-magnitude complexity reduction.
Within the realm of probabilistic graphical models, message-passing algorithms offer a powerful framework for efficient inference. When dealing with discrete variables, these algorithms essentially amount to the addit...
详细信息
In grant-free non-orthogonal multiple access, the set of active users is unknown a priori. We address the challenging problem of dynamic channel estimation in this context. Assuming that inactive users are completely ...
详细信息
ISBN:
(数字)9781665482431
ISBN:
(纸本)9781665482431
In grant-free non-orthogonal multiple access, the set of active users is unknown a priori. We address the challenging problem of dynamic channel estimation in this context. Assuming that inactive users are completely muted (in order to avoid unwanted pilot/preamble interference on active users), we investigate a channel estimation method performed jointly with user activity detection, multi-user detection and decoding. Leveraging conditional independence assumptions in probabilistic modeling, we introduce a low-complexity channel estimation performing expectation propagation in parallel for the measurements over all receive antennas. The proposed channel estimation technique is evaluated under different receive antenna correlation models and shows a good performance/complexity tradeoff with respect to joint measurement processing.
We present a universally-optimal distributed algorithm for the exact weighted min-cut. The algorithm is guaranteed to complete in (O) over tilde (D + root n) rounds on every graph, recovering the recent result of Dory...
详细信息
ISBN:
(纸本)9781450392624
We present a universally-optimal distributed algorithm for the exact weighted min-cut. The algorithm is guaranteed to complete in (O) over tilde (D + root n) rounds on every graph, recovering the recent result of Dory, Efron, Mukhopadhyay, and Nanongkai [STOC'21], but runs much faster on structured graphs. Specifically, the algorithm completes in (O) over tilde (D) rounds on (weighted) planar graphs or, more generally, any (weighted) excluded-minor family. We obtain this result by designing an aggregation-based algorithm: each node receives only an aggregate of the messages sent to it. While somewhat restrictive, recent work shows any such black-box algorithm can be simulated on any minor of the communication network. Furthermore, we observe this also allows for the addition of (a small number of) arbitrarily-connected virtual nodes to the network. We leverage these capabilities to design a min-cut algorithm that is significantly simpler compared to prior distributed work. We hope this paper showcases how working within this paradigm yields simple-to-design and ultra-efficient distributed algorithms for global problems. Our main technical contribution is a distributed algorithm that, given any tree T, computes the minimum cut that 2-respects T (i.e., cuts at most 2 edges of T) in universally near-optimal time. Moreover, our algorithm gives a deterministic (O) over tilde (D)-round 2-respecting cut solution for excluded-minor families and a deterministic (O) over tilde (D + root n)-round solution for general graphs, the latter resolving a question of Dory, et al. [STOC'21]
We study the behavior of the belief-propagation (BP) algorithm affected by erroneous data exchange in a wireless sensor network (WSN). The WSN conducts a distributed multidimensional hypothesis test over binary random...
详细信息
We study the behavior of the belief-propagation (BP) algorithm affected by erroneous data exchange in a wireless sensor network (WSN). The WSN conducts a distributed multidimensional hypothesis test over binary random variables. The joint statistical behavior of the sensor observations is modeled by a Markov random field whose parameters are used to build the BP messages exchanged between the sensing nodes. Through linearization of the BP message-update rule, we analyze the behavior of the resulting erroneous decision variables and derive closed-form relationships that describe the impact of stochastic errors on the performance of the BP algorithm. We then develop a decentralized distributed optimization framework to enhance the system performance by mitigating the impact of errors via a distributed linear data-fusion scheme. Finally, we compare the results of the proposed analysis with the existing works and visualize, via computer simulations, the performance gain obtained by the proposed optimization.
暂无评论