variational autoencoder (VAE) is a popular deep generative framework with a solid theoretical basis. There are many research efforts on improving VAE. Among the existing works, a recently proposed deterministic Regula...
详细信息
ISBN:
(纸本)9781450384469
variational autoencoder (VAE) is a popular deep generative framework with a solid theoretical basis. There are many research efforts on improving VAE. Among the existing works, a recently proposed deterministic Regularized autoencoder (RAE) provides a new scheme for generative modeling. RAE fixes the variance of the inferred Gaussian approximate posterior distribution as a hyperparameter, and substitutes the stochastic encoder by injecting noise into the input of a deterministic decoder. However, the deterministic RAE has three limitations: 1) RAE needs to fit the variance;2) RAE requires ex-post density estimation to ensure sample quality;3) RAE employs an additional gradient regularization to ensure training smoothness. Thus, it raises an interesting research question: Can we maintain the flexibility of variational inference while simplifying VAE, and at the same time ensuring a smooth training process to obtain good generative performance? Based on the above motivation, in this paper, we propose a novel Semi-deterministic and Contrastive variational Graph autoencoder (SCVG) for item recommendation. The core design of SCVG is to learn the variance of the approximate Gaussian posterior distribution in a semi-deterministic manner by aggregating inferred mean vectors from other connected nodes via graph convolution operation. We analyze the expressive power of SCVG for the Weisfeiler-Lehman graph isomorphism test, and we deduce the simplified form of the evidence lower bound of SCVG. Besides, we introduce an efficient contrastive regularization instead of gradient regularization. We empirically show that the contrastive regularization makes learned user/item latent representation more personalized and helps to smooth the training process. We conduct extensive experiments on three real-world datasets to show the superiority of our model over state-of-the-art methods for the item recommendation task. Codes are available at https://***/syxkason/SCVG.
With the increase of uncertain elements in power systems and extensive deployment of online monitoring devices, it is necessary to search a more real-time and robust voltage stability assessment method. This study, us...
详细信息
With the increase of uncertain elements in power systems and extensive deployment of online monitoring devices, it is necessary to search a more real-time and robust voltage stability assessment method. This study, using PMU monitoring data, explores a novel data-driven approach for long-term voltage stability assessment based on variational autoencoder (VAE). Our method is capable of extracting the most representative features by an unsupervised data mining method in a probabilistic learning way. Different from most of familiar feature extraction methods, it regularizes latent features in an expected stochastic distribution. Furthermore, a statistical indicator by sampling latent features after variance reduction is proposed to assess long-term voltage stability. Our approach is tested in various simulated power systems with different load increment models. Other cases show the accuracy and speed of our approach for estimating voltage collapse point. These testing cases successfully demonstrate the accuracy and effectiveness of our approach.
variational autoencoder (VAE) based methods for Collaborative Filtering (CF) demonstrate remarkable performance for one-class (implicit negative) recommendation tasks by extending autoencoders with relaxed but tractab...
详细信息
ISBN:
(纸本)9781450361729
variational autoencoder (VAE) based methods for Collaborative Filtering (CF) demonstrate remarkable performance for one-class (implicit negative) recommendation tasks by extending autoencoders with relaxed but tractable latent distributions. Explicitly modeling a latent distribution over user preferences allows VAEs to learn user and item representations that not only reproduce observed interactions, but also generalize them by leveraging learning from similar users and items. Unfortunately, VAE-CF can exhibit suboptimal learning properties;e.g., VAE-CFs will increase their prediction confidence as they receive more preferences per user, even when those preferences may vary widely and create ambiguity in the user representation. To address this issue, we propose a novel Queryable variational autoencoder (Q-VAE) variant of the VAE that explicitly models arbitrary conditional relationships between observations. The proposed model appropriately increases uncertainty (rather than reduces it) in cases where a large number of user preferences may lead to an ambiguous user representation. Our experiments on two benchmark datasets show that the Q-VAE generally performs comparably or outperforms VAE-based recommenders as well as other state-of-the-art approaches and is generally competitive across the user preference density spectrum, where other methods peak for certain preference density levels.
Multi-market businesses can collect data from different business entities and aggregate data from various sources to create value. However, due to the restriction of privacy regulation, it could be illegal to exchange...
详细信息
ISBN:
(纸本)9781728149615
Multi-market businesses can collect data from different business entities and aggregate data from various sources to create value. However, due to the restriction of privacy regulation, it could be illegal to exchange data between business entities of the same parent company, unless the users have opted-in to allow it. Regulations such as the EU's GDPR allows data exchange if data is anonymized appropriately. In this study, we use variational autoencoder as a mechanism to generate synthetic data. The privacy and utility of the generated data sets are measured. And its performance is compared with the performance of the plain autoencoder. The primary findings of this study are 1) variational autoencoder can be an option for data exchange with good accuracy even when the number of latent dimensions is low 2) plain autoencoder still provides better accuracy when the number of hidden nodes is high 3) variational autoencoder, as a generative model, can be given to a data user to generate his version of data that closely mimic the original data set.
The variational autoencoder(VAE) has been proved to be a most efficient generative model, but its applications in natural language tasks have not been fully developed. A novel variational autoencoder for natural texts...
详细信息
ISBN:
(纸本)9781728119854
The variational autoencoder(VAE) has been proved to be a most efficient generative model, but its applications in natural language tasks have not been fully developed. A novel variational autoencoder for natural texts generation is presented in this paper. Compared to the previously introduced variational autoencoder for natural text where both the encoder and decoder are RNN-based, we propose a new transformer-based architecture and augment the decoder with an LSTM language model layer to fully exploit information of latent variables. We also propose some methods to deal with problems during training time, such as KL divergency collapsing and model degradation. In the experiment, we use random sampling and linear interpolation to test our model. Results show that the generated sentences by our approach are more meaningful and the semantics are more coherent in the latent space.
In recent years, bicycle sharing services (bike-shares) have been established worldwide. One important aspect of bike-share management is to periodically rebalance the positions of the available bikes. Because the bik...
详细信息
ISBN:
(纸本)9781450369640
In recent years, bicycle sharing services (bike-shares) have been established worldwide. One important aspect of bike-share management is to periodically rebalance the positions of the available bikes. Because the bike demand varies by and over time, the number of bikes at each bike-port tends to become unbalanced. To efficiently rebalance a bike-share system, it is essential to predicting the number of bikes in each bike-port. In this paper, we propose a method to predicting bike demand and the number of bike pickups and drop offs at each bike-port every hour, up to 24 hours in advance. To predict demand, we used a time series generation model based on the variational autoencoders model and the Attention based Sequence to Sequence learning model. We named this method "Conditional variational autoencoders considering Partial Future data " (CVAE-PF). In the experiment, our proposed method showed higher prediction accuracy in root mean square error (RMSE) compared to conventional methods.
Recent semantic segmentation systems have achieved significant improvement by performing pixel-wise training with hierarchical features using deep convolutional neural network models. While the learning process usuall...
详细信息
ISBN:
(纸本)9783030306458;9783030306441
Recent semantic segmentation systems have achieved significant improvement by performing pixel-wise training with hierarchical features using deep convolutional neural network models. While the learning process usually requires pixel-level annotated images, it is difficult to get desirable amounts of fine-labeled data and thus the training set size is more likely to be limited, often in thousands. This means that top methods for a dataset can be fine-tuned for a specific situation, making the generalization ability unclear. In real-world applications like self-driving systems, ambiguous region or lack of context information can cause errors in the predicted results. Resolving such ambiguities is crucial for subsequent operations to be performed safely. We are inspired by work from CodeSLAM where optimizable pixel-wise depth representation is learned. We modify the regression method to work on the pixel-wise classification problem. By training a variational auto-encoder network conditioned with a color image, the computed latent space works as a low-dimensional representation of semantic segmentation, which can be efficiently optimized. As a consequence, our model can correct the error or ambiguity of the prediction during the inference phase given useful scene information. We show how this approach works by giving partial scene truth and perform optimization on the latent variable.
Random mixing and circularly shifting for augmenting the training set are used to improve the separation effect of deep neural network (DNN)-based monaural singing voice separation (MSVS). However, these manual method...
详细信息
ISBN:
(纸本)9781538695524
Random mixing and circularly shifting for augmenting the training set are used to improve the separation effect of deep neural network (DNN)-based monaural singing voice separation (MSVS). However, these manual methods are based on unrealistic assumptions that two sources in the mixture are independent of each other, which limits the separation effect. This paper proposes a data augmentation method based on variational autoencoder (VAE) and generative adversarial network (GAN), which is called as VAE-GAN. The VAE models the observed spectra of sources (vocal and music) separately and reconstructs new spectra from the latent space. The GAN's discriminator is introduced to measure the correlation between the latent variables of the vocal and music generated by the VAE probability encoder. This adversarial mechanism in VAE's latent space could learn the synthetic likelihood and ultimately decode high quality spectra samples, which further improves the separation effect of general MSVS networks.
In this work, we investigate the effectiveness of two techniques for improving variational autoencoder (VAE) based voice conversion (VC). First, we reconsider the relationship between vocoder features extracted using ...
详细信息
In this work, we investigate the effectiveness of two techniques for improving variational autoencoder (VAE) based voice conversion (VC). First, we reconsider the relationship between vocoder features extracted using the high quality vocoders adopted in conventional VC systems, and hypothesize that the spectral features are in fact F0 dependent. Such hypothesis implies that during the conversion phase, the latent codes and the converted features in VAE based VC are in fact source F0 dependent. To this end, we propose to utilize the F0 as an additional input of the decoder. The model can learn to disentangle the latent code from the F0 and thus generates converted F0 dependent converted features. Second, to better capture temporal dependencies of the spectral features and the F0 pattern, we replace the frame wise conversion structure in the original VAE based VC framework with a fully convolutional network structure. Our experiments demonstrate that the degree of disentanglement as well as the naturalness of the converted speech are indeed improved.
This paper presents a refinement framework of WaveNet vocoders for variational autoencoder (VAE) based voice conversion (VC), which reduces the quality distortion caused by the mismatch between the training data and t...
详细信息
ISBN:
(纸本)9789082797039
This paper presents a refinement framework of WaveNet vocoders for variational autoencoder (VAE) based voice conversion (VC), which reduces the quality distortion caused by the mismatch between the training data and testing data. Conventional WaveNet vocoders are trained with natural acoustic features but conditioned on the converted features in the conversion stage for VC, and such a mismatch often causes significant quality and similarity degradation. In this work, we take advantage of the particular structure of VAEs to refine WaveNet vocoders with the self-reconstructed features generated by VAE, which are of similar characteristics with the converted features while having the same temporal structure with the target natural features. We analyze these features and show that the self-reconstructed features are similar to the converted features. Objective and subjective experimental results demonstrate the effectiveness of our proposed framework.
暂无评论