We discuss the implementation of two distributed solvers of the random K-SAT problem, based on some development of the recently introduced survey propagation (SP) algorithm. The first solver, called the 'SP diffus...
详细信息
We discuss the implementation of two distributed solvers of the random K-SAT problem, based on some development of the recently introduced survey propagation (SP) algorithm. The first solver, called the 'SP diffusion algorithm', diffuses as dynamical information the maximum bias over the system, so that variable nodes can decide to freeze in a self-organized way, each variable making its decision on the basis of purely local information. The second solver, called the 'SP reinforcement algorithm', makes use of time-dependent external forcing messages on each variable, which are adapted in time in such a way that the algorithm approaches its estimated closest solution. Both methods allow us to find a solution of the random 3-SAT problem in a range of parameters comparable with the best previously described serialized solvers. The simulated time of convergence towards a solution ( if these solvers were implemented on a fully parallel device) grows as log(N).
Modularity is a popular measure of community structure. However, maximizing the modularity can lead to many competing partitions, with almost the same modularity, that are poorly correlated with each other. It can als...
详细信息
Modularity is a popular measure of community structure. However, maximizing the modularity can lead to many competing partitions, with almost the same modularity, that are poorly correlated with each other. It can also produce illusory "communities" in random graphs where none exist. We address this problem by using the modularity as a Hamiltonian at finite temperature and using an efficient belief propagation algorithm to obtain the consensus of many partitions with high modularity, rather than looking for a single partition that maximizes it. We show analytically and numerically that the proposed algorithm works all of the way down to the detectability transition in networks generated by the stochastic block model. It also performs well on real-world networks, revealing large communities in some networks where previous work has claimed no communities exist. Finally we show that by applying our algorithm recursively, subdividing communities until no statistically significant subcommunities can be found, we can detect hierarchical structure in real-world networks more efficiently than previous methods.
We consider the variable selection problem of generalized linear models (GLMs). Stability selection (SS) is a promising method proposed for solving this problem. Although SS provides practical variable selection crite...
详细信息
We consider the variable selection problem of generalized linear models (GLMs). Stability selection (SS) is a promising method proposed for solving this problem. Although SS provides practical variable selection criteria, it is computationally demanding because it needs to fit GLMs to many re-sampled datasets. We propose a novel approximate inference algorithm that can conduct SS without the repeated fitting. The algorithm is based on the replica method of statistical mechanics and vector approximate messagepassing of information theory. For datasets characterized by rotation-invariant matrix ensembles, we derive state evolution equations that macroscopically describe the dynamics of the proposed algorithm. We also show that their fixed points are consistent with the replica symmetric solution obtained by the replica method. Numerical experiments indicate that the algorithm exhibits fast convergence and high approximation accuracy for both synthetic and real-world data.
Ground state entropy of the network source location problem is evaluated at both the replica symmetric level and one-step replica symmetry breaking level using the entropic cavity method. The regime that is a focus of...
详细信息
Ground state entropy of the network source location problem is evaluated at both the replica symmetric level and one-step replica symmetry breaking level using the entropic cavity method. The regime that is a focus of this study, is closely related to the vertex cover problem with randomly quenched covered nodes. The resulting entropic messagepassing inspired decimation and reinforcement algorithms are used to identify the optimal location of sources in single instances of transportation networks. The conventional belief propagation without taking the entropic effect into account is also compared. We find that in the glassy phase the entropic messagepassing inspired decimation yields a lower ground state energy compared to the belief propagation without taking the entropic effect. Using the extremal optimization algorithm, we study the ground state energy and the fraction of frozen hubs, and extend the algorithm to collect statistics of the entropy. The theoretical results are compared with the extremal optimization results.
In this paper, we present a new approach for the analysis of iterative node-based verification-based (NB-VB) recovery algorithms in the context of compressed sensing. These algorithms are particularly interesting due ...
详细信息
In this paper, we present a new approach for the analysis of iterative node-based verification-based (NB-VB) recovery algorithms in the context of compressed sensing. These algorithms are particularly interesting due to their low complexity (linear in the signal dimension n). The asymptotic analysis predicts the fraction of unverified signal elements at each iteration l in the asymptotic regime where n -> l. The analysis is similar in nature to the well-known density evolution technique commonly used to analyze iterative decoding algorithms. To perform the analysis, a message-passing interpretation of NB-VB algorithms is provided. This interpretation lacks the extrinsic nature of standard message-passing algorithms to which density evolution is usually applied. This requires a number of nontrivial modifications in the analysis. The analysis tracks the average performance of the recovery algorithms over the ensembles of input signals and sensing matrices as a function of l. Concentration results are devised to demonstrate that the performance of the recovery algorithms applied to any choice of the input signal over any realization of the sensing matrix follows the deterministic results of the analysis closely. Simulation results are also provided which demonstrate that the proposed asymptotic analysis matches the performance of recovery algorithms for large but finite values of n. Compared to the existing technique for the analysis of NB-VB algorithms, which is based on numerically solving a large system of coupled differential equations, the proposed method is more accurate and simpler to implement.
In grant-free non-orthogonal multiple access, the set of active users is unknown a priori. We address the challenging problem of dynamic channel estimation in this context. Assuming that inactive users are completely ...
详细信息
ISBN:
(数字)9781665482431
ISBN:
(纸本)9781665482431
In grant-free non-orthogonal multiple access, the set of active users is unknown a priori. We address the challenging problem of dynamic channel estimation in this context. Assuming that inactive users are completely muted (in order to avoid unwanted pilot/preamble interference on active users), we investigate a channel estimation method performed jointly with user activity detection, multi-user detection and decoding. Leveraging conditional independence assumptions in probabilistic modeling, we introduce a low-complexity channel estimation performing expectation propagation in parallel for the measurements over all receive antennas. The proposed channel estimation technique is evaluated under different receive antenna correlation models and shows a good performance/complexity tradeoff with respect to joint measurement processing.
Active learning is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibility of querying a limited number of s...
详细信息
Active learning is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibility of querying a limited number of samples to obtain the corresponding labels, subsequently used for supervised learning. In this work, we consider the task of choosing the subset of samples to be labeled from a fixed finite pool of samples. We assume the pool of samples to be a random matrix and the ground truth labels to be generated by a single-layer teacher random neural network. We employ replica methods to analyze the large deviations for the accuracy achieved after supervised learning on a subset of the original pool. These large deviations then provide optimal achievable performance boundaries for any active learning algorithm. We show that the optimal learning performance can be efficiently approached by simple message-passing active learning algorithms. We also provide a comparison with the performance of some other popular active learning strategies.
We present a universally-optimal distributed algorithm for the exact weighted min-cut. The algorithm is guaranteed to complete in (O) over tilde (D + root n) rounds on every graph, recovering the recent result of Dory...
详细信息
ISBN:
(纸本)9781450392624
We present a universally-optimal distributed algorithm for the exact weighted min-cut. The algorithm is guaranteed to complete in (O) over tilde (D + root n) rounds on every graph, recovering the recent result of Dory, Efron, Mukhopadhyay, and Nanongkai [STOC'21], but runs much faster on structured graphs. Specifically, the algorithm completes in (O) over tilde (D) rounds on (weighted) planar graphs or, more generally, any (weighted) excluded-minor family. We obtain this result by designing an aggregation-based algorithm: each node receives only an aggregate of the messages sent to it. While somewhat restrictive, recent work shows any such black-box algorithm can be simulated on any minor of the communication network. Furthermore, we observe this also allows for the addition of (a small number of) arbitrarily-connected virtual nodes to the network. We leverage these capabilities to design a min-cut algorithm that is significantly simpler compared to prior distributed work. We hope this paper showcases how working within this paradigm yields simple-to-design and ultra-efficient distributed algorithms for global problems. Our main technical contribution is a distributed algorithm that, given any tree T, computes the minimum cut that 2-respects T (i.e., cuts at most 2 edges of T) in universally near-optimal time. Moreover, our algorithm gives a deterministic (O) over tilde (D)-round 2-respecting cut solution for excluded-minor families and a deterministic (O) over tilde (D + root n)-round solution for general graphs, the latter resolving a question of Dory, et al. [STOC'21]
We consider the phase retrieval problem, in which the observer wishes to recover a n-dimensional real or complex signal X-star from the (possibly noisy) observation of vertical bar Phi X-star vertical bar, in which Ph...
详细信息
We consider the phase retrieval problem, in which the observer wishes to recover a n-dimensional real or complex signal X-star from the (possibly noisy) observation of vertical bar Phi X-star vertical bar, in which Phi is a matrix of size m x n. We consider a high-dimensional setting where n, m -> infinity with m/n = O(1), and a large class of (possibly correlated) random matrices Phi and observation channels. Spectral methods are a powerful tool to obtain approximate observations of the signal X-star which can be then used as initialization for a subsequent algorithm, at a low computational cost. In this paper, we extend and unify previous results and approaches on spectral methods for the phase retrieval problem. More precisely, we combine the linearization of message-passing algorithms and the analysis of the Bethe Hessian, a classical tool of statistical physics. Using this toolbox, we show how to derive optimal spectral methods for arbitrary channel noise and right-unitarily invariant matrix Phi, in an automated manner (i.e. with no optimization over any hyperparameter or preprocessing function).
A reliability-based message-passing schedule for iterative decoding of low-density parity-check codes is proposed. Simulation results for bit-flipping algorithms (with binary messages) show that reliability-based sche...
详细信息
ISBN:
(纸本)0780385330
A reliability-based message-passing schedule for iterative decoding of low-density parity-check codes is proposed. Simulation results for bit-flipping algorithms (with binary messages) show that reliability-based schedule can provide considerable improvement in performance and decoding speed over the so-called flooding (parallel) schedule as well as the existing graph-based schedules. The cost associated with this improvement is negligible and is equivalent to having a 2-bit representation for initial messages, instead of the standard 1-bit for hard-decision algorithms, only at the first iteration (all the exchanged messages are still binary).
暂无评论