Feature selection plays an important role in pattern classification. It is especially an important preprocessing task when there are large number of features in comparison to number of patterns as is the case with gen...
详细信息
ISBN:
(纸本)9781479919598
Feature selection plays an important role in pattern classification. It is especially an important preprocessing task when there are large number of features in comparison to number of patterns as is the case with gene expression data. A new unsupervised feature selection method has been evolved using autoencoders since autoencoders have the capacity to learn the input features without class information. In order to prevent the autoencoder from overtraining, masking has been used and the reconstruction error of masked input features has been used to compute feature weights in moving average manner. A new aggregation function for autoencoder has also been introduced by incorporating correlation between input features to remove the redundancy in selected features set. Comparative performance evaluation on benchmark image and gene expression datasets shows that the proposed method outperforms other unsupervised feature selection methods.
Modern software development is highly dependent on existing libraries, frameworks and tools. Finding and learning the ones best suited to solve a given problem can sometimes take a considerable amount of time. Search ...
详细信息
ISBN:
(纸本)9781614996743;9781614996736
Modern software development is highly dependent on existing libraries, frameworks and tools. Finding and learning the ones best suited to solve a given problem can sometimes take a considerable amount of time. Search for appropriate repositories is often done using keywords and standard Web search engines. In this paper we present an alternative way of searching software repositories, based on repository similarity. We have obtained Github repository metadata and constructed a Deep Neural Network, using a Variational autoencoder, that learns a simplified signature, i.e. latent variable model, of each project. Relying on such simplified representation, we have made a system that can easily obtain similar projects based on the Euclidean distance between the latent variables. We provide a 2D project map of projects constructed based on projects similarity. Our system can also generate metadata for projects that do not exist yet, in order to provide suggestions for future software development.
In this paper, we present an efficient page segmentation method for historical document images. Many existing methods either rely on hand-crafted features or perform rather slow as they treat the problem as a pixel-le...
详细信息
ISBN:
(纸本)9781509017928
In this paper, we present an efficient page segmentation method for historical document images. Many existing methods either rely on hand-crafted features or perform rather slow as they treat the problem as a pixel-level assignment problem. In order to create a feasible method for real applications, we propose to use superpixels as basic units of segmentation, and features are learned directly from pixels. An image is first oversegmented into superpixels with the simple linear iterative clustering (SLIC) algorithm. Then, each superpixel is represented by the features of its central pixel. The features are learned from pixel intensity values with stacked convolutional autoencoders in an unsupervised manner. A support vector machine (SVM) classifier is used to classify superpixels into four classes: periphery, background, text block, and decoration. Finally, the segmentation results are refined by a connected component based smoothing procedure. Experiments on three public datasets demonstrate that compared to our previous method, the proposed method is much faster and achieves comparable segmentation results. Additionally, much fewer pixels are used for classifier training.
Big non-stationary data, which comes in gradual fashion or stream, is one important issue in the application of big data to train deep learning machines. In this paper, we focused on a unique variant of traditional au...
详细信息
ISBN:
(纸本)9781509003631
Big non-stationary data, which comes in gradual fashion or stream, is one important issue in the application of big data to train deep learning machines. In this paper, we focused on a unique variant of traditional autoencoder, which is called Marginalized Linear Stacked Denoising autoencoder (MLSDA). MLSDA uses a simple linear model. It is faster and uses less number of parameters than the traditional SDA. It also takes advantages of convex optimization. It has better improvement in the bag of words feature representation. However, the traditional SDA with stochastic gradient descent has been more widely accepted in many applications. The stochastic gradient descent is naturally an online learning. It makes the traditional SDA more scalable for streaming big data. This paper proposes a simple modification of MLSDA. Our modification uses matrix multiplication concept for online learning. The experiment result showed the similar accuracy level compared with a batch version of MLSDA and using lower computation resources. The online MLSDA will improve the scalability of MLSDA for handling streaming big data that representing bag of words features for natural language processing, information retrieval, and computer vision.
Although deep learning has achieved outstanding performances on several difficult machine learning applications, there are multiple issues that make its application on new problems difficult: speed of training, local ...
详细信息
ISBN:
(纸本)9781467365635
Although deep learning has achieved outstanding performances on several difficult machine learning applications, there are multiple issues that make its application on new problems difficult: speed of training, local minima, and manual selection of hyper-parameters. To overcome these problems, this paper proposes a new evolutionary method, EvoAE, to train autoencoders for deep learning networks. By evolving a population of autoencoders, EvoAE learns multiple features in each autoencoder in the form of hidden nodes, evaluates the autoencoders based on their reconstruction quality, and generates new autoencoders using crossover and mutation with chromosomes made up of hidden nodes and associated connections and weights. EvoAE optimizes network weights and structures of autoencoders simultaneously and employs a mini-batch variant, called Evo-batch, to speed up autoencoder search on large datasets. Furthermore, EvoAE supports different training methods in data partitioning and selection, requires little manual intervention, and reduces overall training time drastically over traditional methods on large datasets.
Learning based hashing plays a pivotal role in largescale visual search. However, most existing hashing algorithms tend to learn shallow models that do not seek representative binary codes. In this paper, we propose a...
详细信息
ISBN:
(纸本)9781467389105
Learning based hashing plays a pivotal role in largescale visual search. However, most existing hashing algorithms tend to learn shallow models that do not seek representative binary codes. In this paper, we propose a novel hashing approach based on unsupervised deep learning to hierarchically transform features into hash codes. Within the heterogeneous deep hashing framework, the autoencoder layers with specific constraints are considered to model the nonlinear mapping between features and binary codes. Then, a Restricted Boltzmann Machine (RBM) layer with constraints is utilized to reduce the dimension in the hamming space. The experiments on the problem of visual search demonstrate the competitiveness of our proposed approach compared to the state of the art.
It has been long debated how the so called cognitive map, the set of place cells, develops in rat hippocampus. The function of this organ is of high relevance, since the hippocampus is the key component of the medial ...
详细信息
It has been long debated how the so called cognitive map, the set of place cells, develops in rat hippocampus. The function of this organ is of high relevance, since the hippocampus is the key component of the medial temporal lobe memory system, responsible for forming episodic memory, declarative memory, the memory for facts and rules that serve cognition in humans. Here, a general mechanism is put forth: We introduce the novel concept of Cartesian factors. We show a non-linear projection of observations to a discretized representation of a Cartesian factor in the presence of a representation of a complementing one. The computational model is demonstrated for place cells that we produce from the egocentric observations and the head direction signals. Head direction signals make the observed factor and sparse allothetic signals make the complementing Cartesian one. We present numerical results, connect the model to the neural substrate, and elaborate on the differences between this model and other ones, including Slow Feature Analysis [17].
We present a new deep neural network architecture, motivated by sparse random matrix theory that uses a low-complexity embedding through a sparse matrix instead of a conventional stacked autoencoder. We regard autoenc...
详细信息
ISBN:
(数字)9783319466811
ISBN:
(纸本)9783319466811;9783319466804
We present a new deep neural network architecture, motivated by sparse random matrix theory that uses a low-complexity embedding through a sparse matrix instead of a conventional stacked autoencoder. We regard autoencoders as an information-preserving dimensionality reduction method, similar to random projections in compressed sensing. Thus, exploiting recent theory on sparse matrices for dimensionality reduction, we demonstrate experimentally that classification performance does not deteriorate if the autoencoder is replaced with a computationally-efficient sparse dimensionality reduction matrix.
Early diagnosis, playing an important role in preventing progress and treating the Alzheimer's disease (AD), is based on classification of features extracted from brain images. The features have to accurately capt...
详细信息
ISBN:
(纸本)9781467399616
Early diagnosis, playing an important role in preventing progress and treating the Alzheimer's disease (AD), is based on classification of features extracted from brain images. The features have to accurately capture main AD-related variations of anatomical brain structures, such as, e.g., ventricles size, hippocampus shape, cortical thickness, and brain volume. This paper proposed to predict the AD with a deep 3D convolutional neural network (3D-CNN), which can learn generic features capturing AD biomarkers and adapt to different domain datasets. The 3D-CNN is built upon a 3D convolutional autoencoder, which is pre-trained to capture anatomical shape variations in structural brain MRI scans. Fully connected upper layers of the 3D-CNN are then fine-tuned for each task-specific AD classification. Experiments on the CADDementia MRI dataset with no skull-stripping preprocessing have shown our 3D-CNN outperforms several conventional classifiers by accuracy. Abilities of the 3D-CNN to generalize the features learnt and adapt to other domains have been validated on the ADNI dataset.
The stacked autoencoder is a deep learning model that consists of multiple autoencoders. This model has been widely applied in numerous machine learning applications. A significant amount of effort has been made to in...
详细信息
ISBN:
(纸本)9781509001545
The stacked autoencoder is a deep learning model that consists of multiple autoencoders. This model has been widely applied in numerous machine learning applications. A significant amount of effort has been made to increase the size of the deep learning model with respect to the size of the training dataset and the parameter of the model to improve performance. However, training a large deep learning model is highly time consuming. Recent studies have applied the CPU cluster with thousands of machines as well as the single GPU or the GPU cluster, to train large scale deep learning models. As a high-performance coprocessor like GPU, the Xeon Phi can be an alternative tool for training large scale deep learning models on a single machine. The Xeon Phi can be recognized as a small cluster which features about 60 cores, and each core supports four hardware threads. Massive parallelism offsets the low computing capacity of every core, but challenges an efficient parallel autoencoders design. In this paper, we analyze the training algorithm of autoencoders based on the matrix operation and point out the thread oversubscription problem, which results in performance degradation. Based on the observation, we propose our map-reduce implementation of autoencoders on the Xeon Phi coprocessor. Our basic idea is to parallelize multiple autoencoder model replicas with bulk synchronous parallel (BSP) communication model where the parameters are updated after the computations of all replicas are completed. Each thread is responsible for one model replica, and all replicas work together on the same mini-batch. This data parallelism method is suitable for training autoencoders on the Xeon Phi, and can extend to asynchronous parallel training method without thread oversubscription. In our experiment the speedup is four times higher than that of sequential implementation. Enlarging the size of the autoencoder model, our method still gets stable speedup.
暂无评论