probabilistic Circuits (PCs) are prominent tractable probabilisticmodels, allowing for a wide range of exact inferences. This paper focuses on the main algorithm for training PCs, LearnSPN, a gold standard due to its...
详细信息
probabilistic Circuits (PCs) are prominent tractable probabilisticmodels, allowing for a wide range of exact inferences. This paper focuses on the main algorithm for training PCs, LearnSPN, a gold standard due to its efficiency, performance, and ease of use, in particular for tabular data. We show that LearnSPN is a greedy likelihood maximizer under mild assumptions. While inferences in PCs may use the entire circuit structure for processing queries, LearnSPN applies a hard method for learning them, propagating at each sum node a data point through one and only one of the children/edges as in a hard clustering process. We propose a new learning procedure named SoftLearn, that induces a PC using a soft clustering process. We investigate the effect of this learning-inference compatibility in PCs. Our experiments show that SoftLearn outperforms LearnSPN in many situations, yielding better likelihoods and arguably better samples. We also analyze comparable tractable models to highlight the differences between soft/hard learning and model querying.
Native Language Identification (NLI) is the task of identifying the native language of an author of a text written in a second language. Support Vector Machines and Maximum Entrophy Learners are the most common method...
详细信息
ISBN:
(纸本)9781479922994
Native Language Identification (NLI) is the task of identifying the native language of an author of a text written in a second language. Support Vector Machines and Maximum Entrophy Learners are the most common methods used to solve this problem, but we consider it from the point-of-view of probabilisticgraphicalmodels. We hypothesize that graphicalmodels are well-suited to this task, as they can capture feature inter-dependencies that cannot be exploited by SVMs. Using progressively more connected graphicalmodels, we show that these models out-perform SVMs on reduced feature sets. Furthermore, on full feature sets, even naive Bayes increases accuracy from 82.06% to 83.41% over SVMs on a 5-language classification task.
Under the tuple-level uncertainty paradigm, we introduce a novel graphical model, Generator-Recognizer Network (GRN), as a model for probabilistic databases. The GRN modeling framework extends existing graphical model...
详细信息
ISBN:
(纸本)9781424454440
Under the tuple-level uncertainty paradigm, we introduce a novel graphical model, Generator-Recognizer Network (GRN), as a model for probabilistic databases. The GRN modeling framework extends existing graphicalmodels of probabilistic databases and is capable of representing a much wider range of dependence structures.
Lifted probabilistic inference exploits symmetries in probabilisticgraphicalmodels to allow for tractable probabilistic inference with respect to domain sizes. To exploit symmetries in, e.g., factor graphs, it is cr...
详细信息
Lifted probabilistic inference exploits symmetries in probabilisticgraphicalmodels to allow for tractable probabilistic inference with respect to domain sizes. To exploit symmetries in, e.g., factor graphs, it is crucial to identify commutative factors, i.e., factors having symmetries within themselves due to their arguments being exchangeable. The current state of the art to check whether a factor is commutative with respect to a subset of its arguments iterates over all possible subsets of the factor's arguments, i.e., O(2(n)) iterations for a factor with n arguments in the worst case. In this paper, we efficiently solve the problem of detecting commutative factors in a factor graph. In particular, we introduce the detection of commutative factors (DECOR) algorithm, which allows us to drastically reduce the computational effort for checking whether a factor is commutative in practice. We prove that DECOR efficiently identifies restrictions to drastically reduce the number of required iterations and validate the efficiency of DECOR in our empirical evaluation.
We propose a novel graphical model for probabilistic image segmentation that contributes both to aspects of perceptual grouping in connection with image segmentation, and to globally optimal inference with higher-orde...
详细信息
ISBN:
(纸本)9781457711022
We propose a novel graphical model for probabilistic image segmentation that contributes both to aspects of perceptual grouping in connection with image segmentation, and to globally optimal inference with higher-order graphicalmodels. We represent image partitions in terms of cellular complexes in order to make the duality between connected regions and their contours explicit. This allows us to formulate a graphical model with higher-order factors that represent the requirement that all contours must be closed. The model induces a probability measure on the space of all partitions, concentrated on perceptually meaningful segmentations. We give a complete polyhedral characterization of the resulting global inference problem in terms of the multicut polytope and efficiently compute global optima by a cutting plane method. Competitive results for the Berkeley segmentation benchmark confirm the consistency of our approach.
In this paper we introduce a general probabilisticgraphical model for human everyday activity recognition. The proposed model is a discriminative graphical model with hidden variables for modeling body pose and seque...
详细信息
ISBN:
(纸本)9781509058204
In this paper we introduce a general probabilisticgraphical model for human everyday activity recognition. The proposed model is a discriminative graphical model with hidden variables for modeling body pose and sequential order of them. We use a unified framework for prediction task that is faster and more efficient than structured support vector machine and hidden conditional random fields. We have trained and tested the model on RGB-D videos and the result was comparable to the state of the art.
Sum-Product Networks (SPNs) are hierarchical, graphicalmodels that combine benefits of deep learning and probabilistic modeling. SPNs offer unique advantages to applications demanding exact probabilistic inference ov...
详细信息
Sum-Product Networks (SPNs) are hierarchical, graphicalmodels that combine benefits of deep learning and probabilistic modeling. SPNs offer unique advantages to applications demanding exact probabilistic inference over high-dimensional, noisy inputs. Yet, compared to convolutional neural nets, they struggle with capturing complex spatial relationships in image data. To alleviate this issue, we introduce Deep Generalized Convolutional Sum-Product Networks (DGC-SPNs), which encode spatial features in a way similar to CNNs, while preserving the validity of the probabilistic SPN model. As opposed to existing SPN-based image representations, DGC-SPNs allow for overlapping convolution patches through a novel parameterization of dilations and strides, resulting in significantly improved feature coverage and feature resolution. DGC-SPNs substantially outperform other SPN architectures across several visual datasets and for both generative and discriminative tasks, including image inpainting and classification. These contributions are reinforced by the first simple, scalable, and GPU-optimized implementation of SPNs, integrated with the widely used Keras/TensorFlow framework. The resulting model is fully probabilistic and versatile, yet efficient and straightforward to apply in practical applications in place of traditional deep nets.
The Estimation of Distribution Algorithms (EDAs) is a novel class of evolutionary algorithms which is motivated by the idea of building probabilisticgraphical model of promising solutions to represent linkage informa...
详细信息
ISBN:
(纸本)9781457715846
The Estimation of Distribution Algorithms (EDAs) is a novel class of evolutionary algorithms which is motivated by the idea of building probabilisticgraphical model of promising solutions to represent linkage information between variables in chromosome. Through learning of and sampling from probabilisticgraphical model, new population is generated and optimization procedure is repeated until the stopping criteria are met. In this paper, the mechanism of the Estimation of Distribution Algorithms is analyzed. Currently existing EDAs are surveyed and categorized according to the probabilistic model they used, then the strengths and weakness and the future perspective of EDAs are concluded.
Efficient tracking of class performance across topics is an important aspect of classroom teaching;this is especially true for psychometric general intelligence exams, which test a varied range of abilities. We develo...
详细信息
ISBN:
(纸本)9781450342391
Efficient tracking of class performance across topics is an important aspect of classroom teaching;this is especially true for psychometric general intelligence exams, which test a varied range of abilities. We develop a framework that uncovers a hidden thematic structure underlying student responses to a large pool of questions, using a probabilisticgraphical model.
We design and implement a scalable version of loopy belief propagation (BP), a widely used algorithm for performing inference on probabilisticgraphicalmodels. However, implementations of BP on generic data processin...
详细信息
ISBN:
(纸本)9781538627150
We design and implement a scalable version of loopy belief propagation (BP), a widely used algorithm for performing inference on probabilisticgraphicalmodels. However, implementations of BP on generic data processing platforms such as Apache Spark do not scale well to very large graphicalmodels containing billions of vertices. To handle such large-scale graphs, we leverage a number of strategies. Our implementation is based on Apache Spark GraphX. We propose a novel graph partitioning strategy to reduce both computation and communication overhead providing a 2x speed-up. We use efficient memory management for storing the graph and shared memory for highspeed communication. To evaluate performance and demonstrate scalability of the approach, we perform a range of experiments including using real-world graphs with billions of vertices, where we achieve an overall 10x speed-up over a vanilla Spark baseline. Further, we apply our BP implementation to infer the probability of a website being malicious by performing inference on a graphical model derived from real, large-scale hyperlinked web-crawl data. We have open sourced our implementation.
暂无评论