We present a model-agnostic framework for jointly optimizing the predictive performance and interpretability of supervised machine learning models for tabular data. Interpretability is quantified via three measures: f...
详细信息
ISBN:
(纸本)9798400701191
We present a model-agnostic framework for jointly optimizing the predictive performance and interpretability of supervised machine learning models for tabular data. Interpretability is quantified via three measures: feature sparsity, interaction sparsity of features, and sparsity of non-monotone feature effects. By treating hyperparameter optimization of a machine learning algorithm as a multi-objective optimization problem, our framework allows for generating diverse models that trade off high performance and ease of interpretability in a single optimization run. Efficient optimization is achieved via augmentation of the search space of the learning algorithm by incorporating feature selection, interaction and monotonicity constraints into the hyperparameter search space. We demonstrate that the optimization problem effectively translates to finding the Pareto optimal set of groups of selected features that are allowed to interact in a model, along with finding their optimal monotonicity constraints and optimal hyperparameters of the learning algorithm itself. We then introduce a novel evolutionary algorithm that can operate efficiently on this augmented search space. In benchmark experiments, we show that our framework is capable of finding diverse models that are highly competitive or outperform state-of-the-art XGBoost or Explainable Boosting Machine models, both with respect to performance and interpretability.
Denoising autoencoder genetic programming (DAE-GP) is an estimation of distribution genetic programming (EDA-GP) algorithm. It uses denoising autoencoder long short-term memory networks as probabilistic model to repla...
详细信息
ISBN:
(纸本)9798400701207
Denoising autoencoder genetic programming (DAE-GP) is an estimation of distribution genetic programming (EDA-GP) algorithm. It uses denoising autoencoder long short-term memory networks as probabilistic model to replace the standard mutation and recombination operators of genetic programming (GP). Recent work has shown several advantages regarding solution length and overall performance of DAE-GP when compared to GP. However, training a neural network at each generation is computationally expensive, where model training is the most time consuming process of DAE-GP. In this work, we propose pretraining to reduce the runtime of the DAE-GP. In pretraining, the neural network is trained preceding the evolutionary search. In experiments on 8 real-world symbolic regression tasks we find that DAE-GP with pretraining has a reduced overall runtime of an order of magnitude while generating individuals with similar or better fitness.
In this paper, an efficient method is proposed for short-term load forecasting (STLF) in power systems. The proposed method integrates Generalized Radial Basis Function Network (GRBFN) of Artificial Neural Network (AN...
详细信息
ISBN:
(纸本)9781713872344
In this paper, an efficient method is proposed for short-term load forecasting (STLF) in power systems. The proposed method integrates Generalized Radial Basis Function Network (GRBFN) of Artificial Neural Network (ANN) with Autoencoder of pretraining in Deep Neural Network (DNN). GRBFN is an extension of Radial Basis Function Network (RBFN) in a way that the parameters of the Gaussian function are determined by the learning process. Autoencoder plays an important role to reduce the number of input variables, which means the dimensionality reduction due to the feature extraction. The hybrid model of GRBFN and Autoencoder results in DNN to improve forecasting model accuracy. Also, evolutionary Particle Swarm Optimization (EPSO) of evolutionary computation is employed to optimize the parameters of GRBFN. The proposed method is successfully applied to real data of short-term load forecasting. Copyright (c) 2023 The Authors.
Multi-objective optimization problems are commonplace in real-world applications, and evolutionary algorithms are successful in solving them. Baby search algorithm is a novel evolutionary algorithm proposed recently, ...
详细信息
ISBN:
(纸本)9783031366215;9783031366222
Multi-objective optimization problems are commonplace in real-world applications, and evolutionary algorithms are successful in solving them. Baby search algorithm is a novel evolutionary algorithm proposed recently, which has excellent ability on exploration and exploitation. However, it is designed to cater to single-objective optimization problems, but in this paper, we expand and modify it for multi-objective optimization. We introduce the boundary selection strategy to choose individuals from the Pareto archive for generating new solutions. To determine the best position of each individual we combine Pareto domination relation with random selection. Additionally, we propose an adapted Levy flight method to find promising solutions. Eleven standard multi-objective testing instances, five prevailing indicators and five state-of-art algorithms are applied to evaluate our algorithm. Experiments results demonstrate that our algorithm performs well on IGD, HV, Spread and GD measures.
The design of Boolean functions which exhibit high-quality cryptography properties is a crucial aspect when implementing secure stream ciphers. To this end, several methods have been proposed to search for secure Bool...
详细信息
ISBN:
(纸本)9798400701207
The design of Boolean functions which exhibit high-quality cryptography properties is a crucial aspect when implementing secure stream ciphers. To this end, several methods have been proposed to search for secure Boolean functions, and, among those, evolutionary algorithms play a prominent role. In this paper, Genetic Programming (GP) is applied for the evolution of Boolean functions in order to maximize one essential property for strong cryptography functions, namely non-linearity. Differently from other approaches, the evolution happens in the space of Walsh Transforms, instead of using a direct representation of the Boolean functions. Specifically, we evolve coefficients of the Walsh Transform to obtain a generic Walsh spectrum, from which it is possible, through spectral inversion, to obtain a pseudo-Boolean function that, consequently, can be mapped to (one of) the nearest Boolean one. Since that function might not be unique, we propose a strategy in which balancedness, another important cryptography property, is preserved as much as possible. To show that the evolutionary search is actually effective in this task, we evolved Boolean functions from 6 to 16 variables. The results show that not only GP is effective in evolving Boolean functions with high non-linearity, but also that balanced functions are discovered.
Global optimization problems are among the most complex and widespread tasks in Computer Science. The capability of finding the global optimum is often hindered by many features-e.g., multi-modality, noisiness and non...
详细信息
ISBN:
(纸本)9798350310177
Global optimization problems are among the most complex and widespread tasks in Computer Science. The capability of finding the global optimum is often hindered by many features-e.g., multi-modality, noisiness and non-differentiability-of the fitness landscape related to the problem. To overcome such issues, Dilation Functions (DFs) can be used to perform problem-dependent manipulations of the fitness landscape to "expand" promising regions and "compress" less promising regions. Since in many real-world scenarios the knowledge of the problem characteristics to handcraft tailored DFs is lacking, two automatic approaches to evolve ad-hoc DFs have been proposed and assessed on benchmark problems. One approach is a two-layered method that leverages an Evolution Strategies (ES) and a self-tuning variant of Particle Swarm Optimization to evolve a DF. The other approach uses Genetic Programming (GP) to evolve a set of tailored DFs for each dimension of the search space. In this work, we introduce evolutionary LBDFs (EvLBDFs), a novel approach based on ES to evolve Local Bubble Dilation Functions, a family of DFs that locally dilate hyper-spherical bounded regions of the search space. Moreover, we compare these approaches to solve the Parameter Estimation (PE) problems of two bio-chemical systems. Our results highlight that all three approaches evolved DFs that simplified the PE landscapes. The GP-based approach outperformed the other approaches on the PE problem with the higher number of kinetic parameters to infer.
The automatic construction of an image filter is a difficult task for which many recent machine-learning methods have been proposed. Cartesian Genetic Programming (CGP) has been effectively used in image-processing ta...
详细信息
ISBN:
(纸本)9798400701207
The automatic construction of an image filter is a difficult task for which many recent machine-learning methods have been proposed. Cartesian Genetic Programming (CGP) has been effectively used in image-processing tasks by evolving programs with a function set specialized for computer vision. Although standard CGP is able to construct understandable image filter programs, we hypothesize that explicitly using a mechanism to control the size of the generated filter programs would help reduce the size of the final solution while keeping comparable efficacy on a given task. It is indeed central to keep the graph size as contained as possible as it improves our ability to understand them and explain their inner functioning. In this work, we use the Lexicase selection as the mechanism to control the size of the programs during the evolutionary process, by allowing CGP to evolve solutions based on performance and on the size of such solutions. We extend Kartezio, a Cartesian Genetic Programming for computer vision tasks, to generate our programs. We found in our preliminary experiment that CGP with Lexicase selection is able to achieve similar performance to the standard CGP while keeping the size of the solutions smaller.
Urban skyways, in which an elevated pedestrian-friendly layer of the city is applied to the existing urban fabric, have evolved from radical conceptual proposals in the mid-twentieth century, such as the continuous mo...
详细信息
Generating new instances via evolutionary methods is commonly used to create newbenchmarking data-sets, with a focus on attempting to cover an instance-space as completely as possible. Recent approaches have exploited...
详细信息
ISBN:
(纸本)9798400701191
Generating new instances via evolutionary methods is commonly used to create newbenchmarking data-sets, with a focus on attempting to cover an instance-space as completely as possible. Recent approaches have exploited Quality-Diversity methods to evolve sets of instances that are both diverse and discriminatory with respect to a portfolio of solvers, but these methods can be challenging when attempting to find diversity in a high-dimensional feature-space. We address this issue by training a model based on Principal Component Analysis on existing instances to create a low-dimension projection of the high-dimension feature-vectors, and then apply Novelty Search directly in the new low-dimension space. We conduct experiments to evolve diverse and discriminatory instances of Knapsack Problems, comparing the use of Novelty Search in the original feature-space to using Novelty Search in a low-dimensional projection, and repeat over a given set of dimensions. We find that the methods are complementary: if treated as an ensemble, they collectively provide increased coverage of the space. Specifically, searching for novelty in a low-dimension space contributes 56% of the filled regions of the space, while searching directly in the feature-space covers the remaining 44%.
This paper aims to provide an introduction to genetic algorithms and their three main components, i.e., the representation of solutions and their modification through mutation and crossover operators. It has been spec...
详细信息
This paper aims to provide an introduction to genetic algorithms and their three main components, i.e., the representation of solutions and their modification through mutation and crossover operators. It has been specifically designed as introduction for newcomers to this exciting research area. This short paper represents a summary of the full paper found online in IEEE Xplore. The latter provides interactive components for a hands-on exploration of the covered material.
暂无评论