This paper proposes new techniques for data representation in the context of deep learning using agglomerative clustering. Existing autoencoder-based data representation techniques tend to produce a number of encoding...
详细信息
This paper proposes new techniques for data representation in the context of deep learning using agglomerative clustering. Existing autoencoder-based data representation techniques tend to produce a number of encoding and decoding receptive fields of layered autoencoders that are duplicative, thereby leading to extraction of similar features, thus resulting in filtering redundancy. We propose a way to address this problem and show that such redundancy can be eliminated. This yields smaller networks and produces unique receptive fields that extract distinct features. It is also shown that autoencoders with nonnegativity constraints on weights are capable of extracting fewer redundant features than conventional sparse autoencoders. The concept is illustrated using conventional sparse autoencoder and nonnegativity-constrained autoencoders with MNIST digits recognition, NORB normalized-uniform object data and Yale face dataset. (C) 2017 Elsevier Ltd. All rights reserved.
In the simple form, a communication system includes a transmitter and a receiver. In the transmitter, it transforms the one-hot vector message to produce a transmitted signal. In general, the transmitter demands restr...
详细信息
In the simple form, a communication system includes a transmitter and a receiver. In the transmitter, it transforms the one-hot vector message to produce a transmitted signal. In general, the transmitter demands restrictions on the transmitted signal. The channel is defined by the conditional probability distribution function. On receiving of the transmitted signal with noise, the receiver appears to apply the transformation to generate the estimate of one hot vector message. We can regard this simplest communication system as a specific case of autoencoder from a deep learning perspective. In our case, autoencoder used to learn the representations of the one-hot vector which are robust to the noise channel and can be recovered at the receiver with the smallest probability of error. Our task is to make some improvements on the autoencoder systems. We propose different schemes depending on the different cases. We propose a method based on optimization of softmax and introduce the L1/2 regularization in MSE loss function for SISO case and MIMO case, separately. The simulation shows that both our optimized softmax function method and L1/2 regularization loss function have a better performance than the original neural network *** of Applied Science (MASc)
Due to the complexity of modern industrial processes, there may be both linear and nonlinear relationships exist among process variables. In addition, the dynamic behavior of the process also brings challenges to proc...
详细信息
Due to the complexity of modern industrial processes, there may be both linear and nonlinear relationships exist among process variables. In addition, the dynamic behavior of the process also brings challenges to process ***, some linear monitoring methods have been developed for dynamic processes. However, the existing methods can not precisely extract the dynamic characteristics of nonlinear processes. What is more, purely linear or nonlinear methods can hardly tackle the hybrid linear and nonlinear relationships among process variables. To address the above issue, a novel method, termed slow feature networks(SFNet) is proposed and applied for dynamic process monitoring. On the one hand, a slowly varying constraint of hidden features is added to the autoencoder, so that the static and dynamic characteristics of nonlinear processes can be extracted concurrently. On the other hand, a linear mapping is incorporated into the nonlinear neural network structure,thereby providing parallel analysis of linear and nonlinear monitoring information. Five statistics are constructed for comprehensive process monitoring from both static and dynamic, linear and nonlinear perspectives. In this way, alarms corresponding to different statistical information are used to indicate different operating statuses with meaningful interpretation and enhanced process understanding. A real industrial example is adopted to validate the performance of the proposed method.
An average of 8000 forest wildfires occurs each year in Canada burning an average of 2.5M ha/year as reported by the Government of Canada. Given the current rate of climate change, this number is expected to increase ...
详细信息
An average of 8000 forest wildfires occurs each year in Canada burning an average of 2.5M ha/year as reported by the Government of Canada. Given the current rate of climate change, this number is expected to increase each year. Being able to predict how the fires spread would play a critical role in fire risk management. However, given the complexity of the natural processes that influence a fire system, most of the models used for simulating wildfires are computationally expensive and need a high variety of information about the environmental parameters to be able to give good performances. Deep learning algorithms allow computers to learn from experience and understand the world in terms of a hierarchy of concepts, with each concept defined in terms of its relation to simpler concepts. We propose a deep learning predictor that uses a Deep Convolutional Auto-Encoder to learn the key structures of a forest wildfire spread from images and a Long Short Term Memory to predict the next phase of the fire. We divided the predictor problem in three phases: find a dataset of wildfires, learning the essential structure of forest fire, and predict the next image. We first present the simulated wildfires dataset and the algorithm we applied on it to make it more suitable to the model. Then we present the Deep Forest Wildfire Auto-Encoder and its implementation using the Caffe framework. Particular attention is given to the design considerations and to the best practice used to implement the model. We also present the design of the Deep Forest Wildfire Predictor, and some possible future variations of it.
This paper investigates the effect of noises added to hidden units of autoencoders linked to multilayer perceptrons. It is shown that internal representation of learned features emerges and sparsity of hidden units in...
详细信息
This paper investigates the effect of noises added to hidden units of autoencoders linked to multilayer perceptrons. It is shown that internal representation of learned features emerges and sparsity of hidden units increases when independent Gaussian noises are added to inputs of hidden units during the deep network training. It is also shown that the weights that connect the contaminated hidden units with the next layer have smaller values and outputs of hidden units tend to be more definite (0 or 1). This is expected to improve the generalization ability of the network through this automatic structuration by adding the noises. This network structuration was confirmed by experiments for MNIST digits classification via a deep neural network model.
Machine learning relies on developing models which represent data in informative and simple ways. Taking inspiration from the subfield of multitask learning, we in- vestigate the possibility of enhancing data represen...
详细信息
Machine learning relies on developing models which represent data in informative and simple ways. Taking inspiration from the subfield of multitask learning, we in- vestigate the possibility of enhancing data representations at intermediate layers in a neural network. Specifically, we add a decoder layer whose task is to reconstruct the model's input from the intermediate representation. Along with this contribu- tion, we introduce a number of algorithms for anomaly detection and supervised classification based on this framework and assess their performance. We find that anomaly detection works best in this framework when formulated as a classification problem between in-distribution and out-of-distribution data, and that supervised classification works best when using the simplest formulation with a linear classifier.
It is almost seventy years after the publication of Claude Shannon's "A Mathematical Theory of Communication" [1] and Norbert Wiener's "Extrapolation, Interpolation and Smoothing of Stationary T...
详细信息
ISBN:
(纸本)9781509041176
It is almost seventy years after the publication of Claude Shannon's "A Mathematical Theory of Communication" [1] and Norbert Wiener's "Extrapolation, Interpolation and Smoothing of Stationary Time Series" [2]. The pioneering works of Shannon and Wiener lay the foundation of communication, data storage, control, and other information technologies. This paper briefly reviews Shannon and Wiener's perspectives on the problem of message transmission over noisy channel and also experimentally evaluates the feasibility of integrating these two perspectives to train autoencoders close to the information limit. To this end, the principle of relevant information (PRI) is used and validated to optimally encode input imagery in the presence of noise.
We demonstrate a novel method for the automatic modulation classification based on a deep learning autoencoder network, trained by a nonnegativity constraint algorithm. The learning algorithm aims to constrain the neg...
详细信息
We demonstrate a novel method for the automatic modulation classification based on a deep learning autoencoder network, trained by a nonnegativity constraint algorithm. The learning algorithm aims to constrain the negative weights, learns features that amount to a part-based representation of data, and disentangles a more meaningful hidden structure. The performance of this algorithm is tested on the fourth-order cumulants of the modulated signals. The results indicate that the autoencoder with nonnegativity constraint (ANC) improves the sparsity and minimizes the reconstruction error in comparison with the conventional sparse autoencoder. The classification accuracy of an ANC based deep network shows improved accuracy under limited signal length and fading channel.
暂无评论