Convolutional Neural Networks for text categorization allows the extraction of features from the text represented through word embedding. The high dimensionality of the texts themselves implies a larger number of netw...
详细信息
Convolutional Neural Networks for text categorization allows the extraction of features from the text represented through word embedding. The high dimensionality of the texts themselves implies a larger number of network parameters and a more complex optimization surface. Artificial neural network training is an NP-Hard optimization problem, which has been addressed by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minimums. In this work, we propose a hybrid method based on the estimation of distribution algorithms for training a Convolutional Neural Network. For this, we train together gradient-based methods with the estimation of Multivariate Normal Algorithm and Univariate Marginal distribution Algorithm by dividing the training process into two stages. The different variants obtained with the proposed method are compared with gradient-based methods on public benchmark datasets and statistical differences are analyzed by nonparametric tests. The proposed method increases the accuracy of the convolutional network applied to the text categorization task and overcome in about 0.22%-24% the state-of-the-art algorithms. (C) 2022 Elsevier B.V. All rights reserved.
Background: Microarray technology allows to measure the expression of thousands of genes simultaneously, and under tens of specific conditions. Clustering and Biclustering are the main tools to analyze gene expression...
详细信息
Background: Microarray technology allows to measure the expression of thousands of genes simultaneously, and under tens of specific conditions. Clustering and Biclustering are the main tools to analyze gene expression data obtained from microarray experiments. By grouping together genes with the same behavior across samples, relevant biological knowledge may be extracted. Non-exclusive groupings are required, since a gene may play more than one biological role. Gene Shaving [Hastie, T., et al. (2000). Gene Shaving as a method for identifying distinct sets of genes with similar expression. Genome Biology, 1, 1-21] is a popular clustering algorithm which looks for coherent clusters of genes with high variance across samples, allowing overlapping among the clusters. Method: In this paper, we present an intelligent system for analyzing microarray data. Our system implements three novel non-exclusive approaches for clustering and biclustering whose aim is to find coherent groups of genes with large between-sample variance: EDA-Clustering and EDA-Biclustering, based on estimation of distribution algorithms (EDA), and Gene-&-Sample Shaving, a biclustering algorithm based on Principal Components Analysis. Results: We integrated the three proposed methods into a web-based platform and tested their performance on two real datasets. The obtained results outperform Gene Shaving in terms of quality and size of revealed patterns. Furthermore, our system allows to visualize the results and validate them from a biological point of view by means of the annotations of the Gene Ontology. (C) 2008 Elsevier Ltd. All rights reserved.
One of the main tasks software testing includes is the generation of the test cases to be used during the test. Due to its expensive cost, the automatization of this task has become one of the key issues in the area. ...
详细信息
One of the main tasks software testing includes is the generation of the test cases to be used during the test. Due to its expensive cost, the automatization of this task has become one of the key issues in the area. The field of Evolutionary Testing deals with this problem by means of metaheuristic search techniques. An Evolutionary Testing based approach to the automatic generation of test inputs is presented. The approach developed involves different possibilities of the usage of two heuristic optimization methods, namely, Scatter Search and estimation of distribution algorithms. The possibilities comprise pure Scatter Search options and Scatter Search-estimation of distribution Algorithm collaborations. Several experiments were conducted in order to evaluate and compare the approaches presented with those in the literature. The analysis of the experimental results raises interesting conclusions, showing these alternatives as a promising option to tackle this problem. (c) 2004 Elsevier B.V. All rights reserved.
Bayesian networks can be used as a model to make inferences in domains with intrinsic uncertainty, that is, to determine the probability distribution of a set of variables given the instantiation of another set. The i...
详细信息
Bayesian networks can be used as a model to make inferences in domains with intrinsic uncertainty, that is, to determine the probability distribution of a set of variables given the instantiation of another set. The inference is an NP-hard problem. There are several algorithms to make exact and approximate inference. One of the most popular, and that is also an exact method. is the evidence propagation algorithm of Lauritzen and Spiegelhalter [S.L. Lauritzen, D.J. Spiegelhalter, Local computations with probabilities on graphical structures and their application on expert systems, journal of the Royal Statistical Society B 50 (2) (1988) 157-224], improved later by Jensen et al. [F.V. Jensen, S.L. Lauritzen, K.G. Olesen, Bayesian updating in causal probabilistic networks by local computations, In Computational Statistics Quaterly 4 (1990) 269-282]. This algorithm needs an ordering of the variables in order to make the triangulation of the moral graph associated with the original Bayesian network structure. The effectiveness of the inference depends on the variable ordering. In this paper, we will use a new paradigm for evolutionary computation, the estimation of distribution algorithms (EDAs), to get the optimal ordering of the variables to obtain the most efficient triangulation. We will also present a new type of evolutionary algorithm, the recursive EDAs (REDAs). We will prove that REDAs improve the behaviour of EDAs in this particular problem, and that their results are competitive with other triangulation techniques. (C) 2008 Elsevier Inc. All rights reserved.
Message passing algorithms (MPAs) have been traditionally used as an inference method in probabilistic graphical models. Some MPA variants have recently been introduced in the field of estimation of distribution algor...
详细信息
Message passing algorithms (MPAs) have been traditionally used as an inference method in probabilistic graphical models. Some MPA variants have recently been introduced in the field of estimation of distribution algorithms (EDAs) as a way to improve the efficiency of these algorithms. Multiple developments on MPAs point to an increasing potential of these methods for their application as part of hybrid EDAs. In this paper we review recent work on EDAs that apply MPAs and propose ways to further extend the useful synergies between MPAs and EDAs. Furthermore, we analyze some of the implications that MPA developments can have in their future application to EDAs and other evolutionary algorithms.
estimation of distribution algorithms (EDAs) are a successful example of how to use machine learning techniques for designing robust and efficient heuristic search algorithms. Understanding the relationship between ED...
详细信息
estimation of distribution algorithms (EDAs) are a successful example of how to use machine learning techniques for designing robust and efficient heuristic search algorithms. Understanding the relationship between EDAs and the space of optimization problems is a fundamental issue for the successful application of this type of algorithms. A step forward in this matter is to create a taxonomy of optimization problems according to the different behaviors that an EDA can exhibit. This paper substantially extends previous work in the proposal of a taxonomy of problems for univariate EDAs, mainly by generalizing those results to EDAs that are able to deal with multivariate dependences among the variables of the problem. Through the definition of an equivalence relation between functions, it is possible to partition the space of problems into equivalence classes in which the algorithm has the same behavior. We provide a sufficient and necessary condition to determine the equivalence between functions. This condition is based on a set of matrices which provides a novel encoding of the relationship between the function and the probabilistic model used by the algorithm. The description of the equivalent functions belonging to a class is studied in depth for EDAs whose probabilistic model is given by a chordal Markov network. Assuming this class of factorization, we unveil the intrinsic connection between the behaviors of EDAs and neighborhood systems defined over the search space. In addition, we carry out numerical simulations that effectively reveal the different behaviors of EDAs for the injective functions defined over the search space (0, 1}(3). Finally, we provide a novel approach to extend the analysis of equivalence classes to non-injective functions. (C) 2015 Elsevier B.V. All rights reserved.
In this paper, a class of continuous estimation of distribution algorithms (EDAs) based on Gaussian models is analyzed to investigate their potential for solving dynamic optimization problems where the global optima m...
详细信息
In this paper, a class of continuous estimation of distribution algorithms (EDAs) based on Gaussian models is analyzed to investigate their potential for solving dynamic optimization problems where the global optima may change dramatically during time. Experimental results on a number of dynamic problems show that the proposed strategy for dynamic optimization can significantly improve the performance of the original EDAs and the optimal solutions can be consistently located.
This article introduces the Coincidence Algorithm (COIN) to solve several multimodal puzzles. COIN is an algorithm in the category of estimation of distribution algorithms (EDAs) that makes use of probabilistic models...
详细信息
This article introduces the Coincidence Algorithm (COIN) to solve several multimodal puzzles. COIN is an algorithm in the category of estimation of distribution algorithms (EDAs) that makes use of probabilistic models to generate solutions. The model of COIN is a joint probability table of adjacent events (coincidence) derived from the population of candidate solutions. A unique characteristic of COIN is the ability to learn from a negative sample. Various experiments show that learning from a negative example helps to prevent premature convergence, promotes diversity and preserves good building blocks.
We perform rigorous runtime analyses for the univariate marginal distribution algorithm (UMDA) and the population-based incremental learning (PBIL) Algorithm on LeadingOnes. For the UMDA, the currently known expected ...
详细信息
We perform rigorous runtime analyses for the univariate marginal distribution algorithm (UMDA) and the population-based incremental learning (PBIL) Algorithm on LeadingOnes. For the UMDA, the currently known expected runtime on the function is O(n lambda log lambda + n(2)) under an offspring population size lambda = Omega(log n) and a parent population size mu <= lambda/(e(1 + delta)) for any constant delta > 0 (Dang and Lehre, GECCO 2015). There is no lower bound on the expected runtime under the same parameter settings. It also remains unknown whether the algorithm can still optimise the LeadingOnes function within a polynomial runtime when mu >= lambda/(e(1+delta)). In case of the PBIL, an expected runtime of O(n(2+c)) holds for some constant c is an element of(0,1) (Wu, Kolonko and Mohring, IEEE TEVC 2017). Despite being a generalisation of the UMDA, this upper bound is significantly asymptotically looser than the upper bound of O(n(2) of the UMDA for lambda=Omega(log n) boolean AND O. Furthermore, the required population size is very large, i.e., lambda=Omega(n(1+c)). Our contributions are then threefold: (1) we show that the UMDA with mu=Omega(log n) and lambda <= mu e(1-epsilon)/(1+delta) for any constants epsilon is an element of(0,1) and 0 < delta <= e(1-epsilon)-1 requires an expected runtime of e(Omega(mu)) on LEADINGONES, (2) an upper bound of O(n lambda log lambda + n(2)) is shown for the PBIL, which improves the current bound O(n(2+c)) by a significant factor of Theta(n(c)), and (3) we for the first time consider the two algorithms on the LeadingOnes function in a noisy environment and obtain an expected runtime of O(n(2)) for appropriate parameter settings. Our results emphasise that despite the independence assumption in the probabilistic models, the UMDA and the PBIL with fine-tuned parameter choices can still cope very well with variable interactions.
estimation of distribution algorithms (EDAs) are widely used in stochastic optimization. Impressive experimental results have been reported in the literature. However, little work has been done on analyzing the comput...
详细信息
estimation of distribution algorithms (EDAs) are widely used in stochastic optimization. Impressive experimental results have been reported in the literature. However, little work has been done on analyzing the computation time of EDAs in relation to the problem size. It is still unclear how well EDAs (with a finite population size larger than two) will scale up when the dimension of the optimization problem (problem size) goes up. This paper studies the computational time complexity of a simple EDA, i.e., the univariate marginal distribution algorithm (UMDA), in order to gain more insight into EDAs complexity. First, we discuss how to measure the computational time complexity of EDAs. A classification of problem hardness based on our discussions is then given. Second, we prove a theorem related to problem hardness and the probability conditions of EDAs. Third, we propose a novel approach to analyzing the computational time complexity of UMDA using discrete dynamic systems and Chernoff bounds. Following this approach, we are able to derive a number of results on the first hitting time of UMDA on a well-known unimodal pseudo-boolean function, i.e., the LeadingOnes problem, and another problem derived from LeadingOnes, named BVLeadingOnes. Although both problems are unimodal, our analysis shows that LeadingOnes is easy for the UMDA, while BVLeadingOnes is hard for the UMDA. Finally, in order to address the key issue of what problem characteristics make a problem hard for UMDA, we discuss in depth the idea of "margins" (or relaxation). We prove theoretically that the UMDA with margins can solve the BVLeadingOnes problem efficiently.
暂无评论