In most of the fault detection methods, the time domain signals collected from the mechanical equipment usually need to be transformed into frequency domain or other high-level data, highly relying on professional kno...
详细信息
In most of the fault detection methods, the time domain signals collected from the mechanical equipment usually need to be transformed into frequency domain or other high-level data, highly relying on professional knowledge such as signal processing and fault pattern recognition. Contrary to those existing approaches, we proposed a two-stage machine learning analysis architecture which can accurately predict the motor fault modes only by using motor vibration time-domain signals without any complicated preprocessing. In the first stage, the method RNN-based VAE was proposed which is highly suitable for dimension reduction of time series data. In addition to reducing the dimension of sequential data from 150*3 to 25 dimensions, our method furthermore improves the prediction accuracy evaluated by several classification algorithms. While other dimension reduction methods such as autoencoder and variational autoencoder cannot improve the classification accuracy effectively or even decreased. It indicates that the sequential data after dimension reduction via the RNN-based VAE still can maintain the high-dimensional data information. Furthermore, the experimental results demonstrate that it can be well applied to time series data dimension reduction and shows a significant improvement of the prediction performance, even with a simple double-layer Neural Network can reach over 99% of accuracy. In the second stage, Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) are used to further perform the second dimension reduction, such that the different or unknown fault modes can be clearly visualized and detected.
variational autoencoders (VAE) represent a popular, flexible form of deep generative model that can be stochastically fit to samples from a given random process using an information-theoretic variational bound on the ...
详细信息
variational autoencoders (VAE) represent a popular, flexible form of deep generative model that can be stochastically fit to samples from a given random process using an information-theoretic variational bound on the true underlying distribution. Once so-obtained, the model can be putatively used to generate new samples from this distribution, or to provide a low-dimensional latent representation of existing samples. While quite effective in numerous application domains, certain important mechanisms which govern the behavior of the VAE are obfuscated by the intractable integrals and resulting stochastic approximations involved. Moreover, as a highly non-convex model, it remains unclear exactly how minima of the underlying energy relate to original design purposes. We attempt to better quantify these issues by analyzing a series of tractable special cases of increasing complexity. In doing so, we unveil interesting connections with more traditional dimensionality reduction models, as well as an intrinsic yet underappreciated propensity for robustly dismissing sparse outliers when estimating latent manifolds. With respect to the latter, we demonstrate that the VAE can be viewed as the natural evolution of recent robust PCA models, capable of learning nonlinear manifolds of unknown dimension obscured by gross corruptions.
Topic models are widely used in various fields of machine learning and statistics. Among them, the dynamic topic model (DTM) is the most popular time-series topic model for the dynamic representations of text corpora....
详细信息
Topic models are widely used in various fields of machine learning and statistics. Among them, the dynamic topic model (DTM) is the most popular time-series topic model for the dynamic representations of text corpora. A major challenge is that the posterior distribution of DTM requires a complex reasoning process with the high cost of computing time in modeling, and even a tiny change of model requires restructuring. For these reasons, the variability and generality of DTM is so poor that DTM is difficult to be carried out. In this paper, we introduce a new method for constructing DTM based on variational autoencoder and factor graphs. This model uses re-parameterization of the variational lower bound to generate a lower bound estimator which is optimized by standard stochastic gradient descent method directly. At the same time, the optimization process is simplified by integrating the dynamic factor graph in the state space to achieve a better model. The experimental dataset uses a journal paper corpus that mainly focuses on natural language processing and spans twenty-five years (1984-2009) from DBLP. Experiment results indicate that the proposed method is effective and feasible by comparing several state-of-the-art baselines.
This paper proposes a deep learning (DL) based method for signal detection in multiple-input multiple-output orthogonal frequency division multiplexing with index modulation (MIMO-OFDM-IM) systems using variational au...
详细信息
This paper proposes a deep learning (DL) based method for signal detection in multiple-input multiple-output orthogonal frequency division multiplexing with index modulation (MIMO-OFDM-IM) systems using variational autoencoder (VAE). A framework of variational optimization is constructed and the learning deep neural network structure with fully connected layers is also designed. The network is trained offline by exploiting the dataset generated by simulations, and then the well-trained model is applied for signal detection in MIMO-OFDM-IM system. The derivation on the loss function for network training is presented from the perspective of variational inference. Also, a regularization parameter is introduced in order to achieve better performance. Numerical results are provided and the performance demonstrates the merits of the proposed method by comparison with the traditional detection algorithms. Finally, complexity measured by the runtime of the program utilizing existing detectors and the proposed method are also presented. The bit error rate (BER) performance which is close to the existing hand-crafted detectors can be obtained with lower runtime using the proposed DL network. (C) 2021 Elsevier Inc. All rights reserved.
Deep neural networks are widely used and exhibit excellent performance in many areas. However, they are vulnerable to adversarial attacks that compromise networks at inference time by applying elaborately designed per...
详细信息
Deep neural networks are widely used and exhibit excellent performance in many areas. However, they are vulnerable to adversarial attacks that compromise networks at inference time by applying elaborately designed perturbations to input data. Although several defense methods have been proposed to address specific attacks, other types of attacks can circumvent these defense mechanisms. Therefore, we propose Purifying variational autoencoder (PuVAE), a method to purify adversarial examples. The proposed method eliminates an adversarial perturbation by projecting an adversarial example on the manifold of each class and determining the closest projection as a purified sample. We experimentally illustrate the robustness of PuVAE against various attack methods without any prior knowledge about the attacks. In our experiments, the proposed method exhibits performances that are competitive with state-of-the-art defense methods, and the inference time is approximately 130 times faster than that of Defense-GAN which is a state-of-the art purifier method.
Sentence generation is a key task in many natural language processing systems. Models based on a variational autoencoder (VAE) can generate plausible sentences from a continuous latent space. However, the VAE forces t...
详细信息
Sentence generation is a key task in many natural language processing systems. Models based on a variational autoencoder (VAE) can generate plausible sentences from a continuous latent space. However, the VAE forces the latent distribution of each input sentence to match the same prior, which results in a large overlap among the latent subspaces of different sentences and a limited informative latent space. Therefore, the sentences generated by sampling from a subspace may have little correlation with the corresponding input, and the latent space cannot capture rich useful information from the input sentences, which leads to the failure of the model to generate diverse sentences from the latent space. Additionally, the Kullback-Leibler (KL) divergence collapse problem makes the VAE notoriously difficult to train. In this paper, a latent space expanded VAE (LSE-VAE) model is presented for sentence generation. The model maps each sentence to a continuous latent subspace under the constraint of its own prior distribution, and constrains nearby sentences to map to nearby subspaces. Sentences are dispersed to a large continuous latent space according to sentence similarity, where the latent subspaces of different sentences may be relatively far away from each other and arranged in an orderly manner. The experimental results show that the LSE-VAE improves the reconstruction ability of the VAE, generates plausible and more diverse sentences, and learns a larger informative latent space than the VAE with the properties of continuity and smoothness. The LSE-VAE does not suffer from the KL collapse problem, and it is robust to hyperparameters and much easier to train.
The ball screw is an important component of machine tools, and its degradation assessment is therefore critical for the health management of the entire machine tool. Generally, the degradation assessment includes heal...
详细信息
The ball screw is an important component of machine tools, and its degradation assessment is therefore critical for the health management of the entire machine tool. Generally, the degradation assessment includes health indicator construction and degradation modeling. However, the health indicator is often constructed manually with prior knowledge, and its sensitivity can be affected by various factors. In addition, most existing degradation models rely on a large amount of failure data, which is not practical for the ball screw due to its high reliability. To solve these problems, this article presents a novel ball screw performance evaluation method. First, the raw data collected in the normal status are used to train the variational autoencoder, and then, the online raw signals are input into the learned variational autoencoder to construct health indicators. After that, the kernel density estimation is utilized to estimate the probability distribution of health indicator points in a dynamic sliding window, and then, the deterioration can be evaluated by summarizing the probability distribution that exceeds a predefined threshold. Experimental results show that the presented methodology can establish the health indicator automatically and adaptively. Also, it can evaluate the ball screw performance effectively and quantitatively when only data in healthy state are available.
Clustering analysis of massive data in wireless multimedia sensor networks (WMSN) has become a hot topic. However, most data clustering algorithms have difficulty in obtaining latent nonlinear correlations of data fea...
详细信息
Clustering analysis of massive data in wireless multimedia sensor networks (WMSN) has become a hot topic. However, most data clustering algorithms have difficulty in obtaining latent nonlinear correlations of data features, resulting in a low clustering accuracy. In addition, it is difficult to extract features from missing or corrupted data, so incomplete data are widely used in practical work. In this paper, the optimally designed variational autoencoder networks is proposed for extracting features of incomplete data and using high-order fuzzy c-means algorithm (HOFCM) to improve cluster performance of incomplete data. Specifically, the feature extraction model is improved by using variational autoencoder to learn the feature of incomplete data. To capture nonlinear correlations in different heterogeneous data patterns, tensor based fuzzy c-means algorithm is used to cluster low-dimensional features. The tensor distance is used as the distance measure to capture the unknown correlations of data as much as possible. Finally, in the case that the clustering results are obtained, the missing data can be restored by using the low-dimensional features. Experiments on real datasets show that the proposed algorithm not only can improve the clustering performance of incomplete data effectively, but also can fill in missing features and get better data reconstruction results.
Recommender systems play an important role in the age of mass information. They allow users to discover items that match their tastes. In this paper, we propose a novel method, called adversarial variational autoencod...
详细信息
ISBN:
(纸本)9781538665657
Recommender systems play an important role in the age of mass information. They allow users to discover items that match their tastes. In this paper, we propose a novel method, called adversarial variational autoencoder, for top-N recommendation. We use generative adversarial networks to regularize variational autoencoder by imposing an arbitrary prior on the latent representation of VAE, which makes the recommendation model. We define a joint objective function as a minimization problem. Our experiments on three datasets show that the proposed model achieves high recommendation accuracy compared to other state-of-the-art models.
Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcem...
详细信息
ISBN:
(纸本)9783030041915;9783030041908
Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming variational autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.
暂无评论