Paradigm-shifting systems such as cyber-physical systems, collect data of high-or ultrahigh-dimensionality tremendously. Detecting outliers in this type of systems provides indicative understanding in wide-ranging dom...
详细信息
ISBN:
(纸本)9781538650356
Paradigm-shifting systems such as cyber-physical systems, collect data of high-or ultrahigh-dimensionality tremendously. Detecting outliers in this type of systems provides indicative understanding in wide-ranging domains such as system health monitoring, information security, etc. Previous dimensionality reduction based outlier detection methods suffer from the incapability of well preserving the critical information in the low-dimensional latent space, mainly because they generally assume an isotropic Gaussian distribution as prior and fail to mine the intrinsic multimodality in high dimensional data. Moreover, most of the schemes decouple the model learning process, resulting in suboptimal performance. To tackle these challenges, in this paper, we propose a unified Unsupervised Gaussian Mixture variational autoencoder for outlier detection. Specifically, a variational autoencoder firstly trains a generative distribution and extracts reconstruction based features. Then we adopt a deep brief network to estimate the component mixture probabilities by the latent distribution and extracted features, which is further used by the Gaussian mixture model to estimate sample densities with the Expectation-Maximization ( EM) algorithm. The inference model is optimized jointly with the variational autoencoder, the deep brief network, and the Gaussian mixture model. Afterwards, the proposed detector identifies outliers when the estimated sample density exceeds a learned threshold. Extensive simulations on six public benchmark datasets show that the proposed framework outperforms state-of-the-art outlier detection schemes and achieves, on average, 27% improvements in F1 score.
This paper proposes a new variational autoencoder (VAE) for topic models. The variational inference (VI) for Bayesian models approximates the true posterior distribution by maximizing a lower bound of the log marginal...
详细信息
ISBN:
(纸本)9783030512521;9783030512538
This paper proposes a new variational autoencoder (VAE) for topic models. The variational inference (VI) for Bayesian models approximates the true posterior distribution by maximizing a lower bound of the log marginal likelihood of observations. We can implement VI as VAE by using a neural network called encoder to obtain parameters of approximate posterior. Our contribution is three-fold. First, we marginalize out per-document topic probabilities by following the proposal by Mimno et al. This marginalizing out has not been considered in the existing VAE proposals for topic modeling to the best of our knowledge. Second, after marginalizing out topic probabilities, we need to approximate the posterior probabilities of token-wise topic assignments. However, this posterior is categorical and thus cannot be approximated by continuous distributions like Gaussian. Therefore, we adopt the Gumbel-softmax trick. Third, while we can sample tokenwise topic assignments with the Gumbel-softmax trick, we should consider document-wide contextual information for a better approximation. Therefore, we feed to our encoder network a concatenation of token-wise information and document-wide information, where the former is implemented as word embedding and the latter as the document-wide mean of word embeddings. The experimental results showed that our VAE improved the existing VAE proposals for a half of the data sets in terms of perplexity or of normalized pairwise mutual information (NPMI).
Grasp planning and most specifically the grasp space exploration is still an open issue in robotics. This article presents a data-driven oriented methodology to model the grasp space of a multi-fingered adaptive gripp...
详细信息
Grasp planning and most specifically the grasp space exploration is still an open issue in robotics. This article presents a data-driven oriented methodology to model the grasp space of a multi-fingered adaptive gripper for known objects. This method relies on a limited dataset of manually specified expert grasps, and uses variational autoencoder to learn grasp intrinsic features in a compact way from a computational point of view. The learnt model can then be used to generate new non-learnt gripper configurations to explore the grasp space. Copyright (C) 2021 The Authors.
Random mixing and circularly shifting for augmenting the training set are used to improve the separation effect of deep neural network (DNN)-based monaural singing voice separation (MSVS). However, these manual method...
详细信息
ISBN:
(纸本)9781538695524
Random mixing and circularly shifting for augmenting the training set are used to improve the separation effect of deep neural network (DNN)-based monaural singing voice separation (MSVS). However, these manual methods are based on unrealistic assumptions that two sources in the mixture are independent of each other, which limits the separation effect. This paper proposes a data augmentation method based on variational autoencoder (VAE) and generative adversarial network (GAN), which is called as VAE-GAN. The VAE models the observed spectra of sources (vocal and music) separately and reconstructs new spectra from the latent space. The GAN's discriminator is introduced to measure the correlation between the latent variables of the vocal and music generated by the VAE probability encoder. This adversarial mechanism in VAE's latent space could learn the synthetic likelihood and ultimately decode high quality spectra samples, which further improves the separation effect of general MSVS networks.
Generating novel molecules with desired properties is a fundamental problem in modern drug discovery. This is a challenging problem because it requires the optimization of the given objectives while obeying the rules ...
详细信息
ISBN:
(纸本)9781450384469
Generating novel molecules with desired properties is a fundamental problem in modern drug discovery. This is a challenging problem because it requires the optimization of the given objectives while obeying the rules of chemical valence. An effective approach is to incorporate the molecular graph with deep generative models. However, recent generative models with high-performance are still computationally expensive. In this paper, we propose GF-VAE, a flow-based variational autoencoder (VAE) model for molecular graph generation. Specifically, the model equips VAE a lightweight flow model as its decoder, in which, the encoder aims to accelerate the training process of the decoder, while the decoder in turns to optimize the performance of the encoder. Thanks to the invertibility of flow model, the generation process is easily accomplished by reversing the decoder. Additionally, the final generated molecules are processed by validity correction. Therefore, our GF-VAE inherits the advantages of both VAE and flow-based methods. We validate our model on molecule generation and reconstruction, smoothness of learned latent space, property optimization and constrained property optimization. The results show that our model achieves state-of-the-arts performance on these tasks. Moreover, the time performance of GF-VAE on two classical datasets can achieve 31.3% and 62.9% improvements separately than the state-of-the-art model.
The hazy weather adversely affects the quality of images from acquisition devices. Traditional image enhancement technology has a poor visual effect. The atmospheric degradation methods combined with prior knowledge a...
详细信息
ISBN:
(纸本)9798350387780;9798350387797
The hazy weather adversely affects the quality of images from acquisition devices. Traditional image enhancement technology has a poor visual effect. The atmospheric degradation methods combined with prior knowledge are required to restore the image under high-scene conditions. As a solution to these problems, we propose an end-to-end deep learning-based dehazing model VDNet based on the variational autoencoder. We design the flexible feature fusion module to prevent the degradation of low-level features during the model training process. In addition, VDNet introduces the perceptual loss, which effectively extracts the features in line with human perception, making the visual effect of dehazing images better. We use the public hazy image dataset RESIDE for model training and testing. Comparative experiments show that VDNet has excellent visual effects and objective indicators of dehazing with a reduced parameter count of 1.971M, which allows it to be used in devices with limited resources. VDNet achieves the result that the average PSNR is 36.98dB and SSIM is 0.9885.
We present a feasibility study for the use of a generative, probabilistic model, a variational autoencoder (VAE), to detect deviations from Standard Model (SM) physics in an electroweak process at the Large Hadron Col...
详细信息
ISBN:
(纸本)9783031510229;9783031510236
We present a feasibility study for the use of a generative, probabilistic model, a variational autoencoder (VAE), to detect deviations from Standard Model (SM) physics in an electroweak process at the Large Hadron Collider (LHC). The new physics responsible for the anomalies is described through an Effective Field Theory (EFT) approach: the SM Lagrangian is Taylor-expanded and the higher order terms cause deviations in the kinematic distributions of the observables, and are thus identified by the model as anomalous contributions with respect to SM. Since the training of the model involves almost only SM events, the proposed strategy is largely independent from any assumption on the nature of the new physics signature. To test the proposed strategy we use parton level generations of Vector Boson Scattering (VBS) events at the LHC, assuming an integrated luminosity of 350 fb(-1).
The successful application of deep neural networks for solving complex tasks like image classification, object detection and segmentation depends critically on the availability of large number of labelled training sam...
详细信息
ISBN:
(纸本)9783031126994;9783031127007
The successful application of deep neural networks for solving complex tasks like image classification, object detection and segmentation depends critically on the availability of large number of labelled training samples. To achieve good generalization for a reasonably complex model with about 60 million parameters, as in AlexNet, one needs about one million labelled training samples. In almost all practical applications, like natural image classification and segmentation, plenty of unlabelled samples are available but labelling these samples is a tedious manual task. We introduce a novel mechanism to automatically label all the samples in an unlabelled dataset. Starting with completely unlabelled dataset, an iterative algorithm incrementally assigns labels along with a confidence to all training samples. During each iteration, 10-30 new representative samples are generated in a latent space learned using a variational autoencoder and labels for these samples are obtained from a human expert. The proposed idea is demonstrated on MNIST dataset without using the labels provided in the dataset. At regular intervals of training, the low dimensional latent vectors are clustered and only cluster centers are annotated. The manual labels of cluster centers are propagated to other samples in the cluster based on the distance and a confidence function. The loss function in successive training is modified to incorporate the manual information provided. We run multiple experiments with different choices of clustering algorithm, confidence function and distance metric and compare the results. With GMM clustering, best classification accuracy of 93.9% was obtained on MNIST test images after 5 iterations.
Magnetometers play a vital role in geophysics and space weather prediction applications by collecting terrestrial magnetic field data. They monitor solar-induced geomagnetic disturbances, providing essential insights ...
详细信息
ISBN:
(纸本)9798350371000;9798350370997
Magnetometers play a vital role in geophysics and space weather prediction applications by collecting terrestrial magnetic field data. They monitor solar-induced geomagnetic disturbances, providing essential insights into predicting space weather effects on technologies such as satellites, power grids, and communication networks. However, this data often contains inherent background noise, necessitating accurate baseline correction methods. Traditional correction techniques are robust but computationally demanding and unsuitable for real-time applications. Recent progress has investigated the utilization of Tiny Machine Learning (TinyML) to process magnetometer data in real time, especially when resources are limited. However, these edge-based ML solutions often lack the robustness of more computationally intensive probabilistic models, such as variational autoencoders (VAEs). This paper introduces a TinyML-VAE surrogate model designed for real-time magnetometer baseline correction. The surrogate model approximates an implemented VAE's performance while operating within the constrained resources of an edge device. The new model retains the VAE's uncertainty quantification capabilities by leveraging surrogate modeling techniques, ensuring robustness. Experimental outcomes have been displayed, illustrating a comparison between the performance of the TinyML-VAE and the benchmark established by the standard VAE.
We propose joint QoT estimation and soft-failure localization leveraging the latent space of a variational autoencoder trained on optical spectra. The framework shows F1-scores of 0.989 for soft-failure detection, 0.9...
详细信息
ISBN:
(纸本)9783903176546
We propose joint QoT estimation and soft-failure localization leveraging the latent space of a variational autoencoder trained on optical spectra. The framework shows F1-scores of 0.989 for soft-failure detection, 0.996 for identification and 0.908 for localization. The QoT estimator reaches an R2-score of 0.998 and a MAE of 0.17 dB.
暂无评论