Topic models are widely used in various fields of machine learning and statistics. Among them, the dynamic topic model (DTM) is the most popular time-series topic model for the dynamic representations of text corpora....
详细信息
Topic models are widely used in various fields of machine learning and statistics. Among them, the dynamic topic model (DTM) is the most popular time-series topic model for the dynamic representations of text corpora. A major challenge is that the posterior distribution of DTM requires a complex reasoning process with the high cost of computing time in modeling, and even a tiny change of model requires restructuring. For these reasons, the variability and generality of DTM is so poor that DTM is difficult to be carried out. In this paper, we introduce a new method for constructing DTM based on variational autoencoder and factor graphs. This model uses re-parameterization of the variational lower bound to generate a lower bound estimator which is optimized by standard stochastic gradient descent method directly. At the same time, the optimization process is simplified by integrating the dynamic factor graph in the state space to achieve a better model. The experimental dataset uses a journal paper corpus that mainly focuses on natural language processing and spans twenty-five years (1984-2009) from DBLP. Experiment results indicate that the proposed method is effective and feasible by comparing several state-of-the-art baselines.
In recent years, Artificial Neural Networks (ANNs) have been introduced in Structural Health Monitoring (SHM) systems. A semi-supervised method with a data-driven approach allows the ANN training on data acquired from...
详细信息
In recent years, Artificial Neural Networks (ANNs) have been introduced in Structural Health Monitoring (SHM) systems. A semi-supervised method with a data-driven approach allows the ANN training on data acquired from an undamaged structural condition to detect structural damages. In standard approaches, after the training stage, a decision rule is manually defined to detect anomalous data. However, this process could be made automatic using machine learning methods. This paper proposes a semi-supervised method with a data-driven approach to detect structural anomalies. The methodology consists of: 1) a variational autoencoder (VAE) to approximate undamaged data distribution and 2) a One-Class Support Vector Machine (OC-SVM) to discriminate different health conditions using damage-sensitive features extracted from VAE's signal reconstruction. The method is applied to a scale steel structure that was tested in nine damage scenarios by IASC-ASCE Structural Health Monitoring Task Group.
In Emotion Recognition in Conversations (ERC), the emotions of target utterances are closely dependent on their context. Therefore, existing works train the model to generate the response of the target utterance, whic...
详细信息
In Emotion Recognition in Conversations (ERC), the emotions of target utterances are closely dependent on their context. Therefore, existing works train the model to generate the response of the target utterance, which aims to recognise emotions leveraging contextual information. However, adjacent response generation ignores long-range dependencies and provides limited affective information in many cases. In addition, most ERC models learn a unified distributed representation for each utterance, which lacks interpretability and robustness. To address these issues, we propose a VAD-disentangled variational autoencoder (VAD-VAE), which first introduces a target utterance reconstruction task based on variational autoencoder, then disentangles three affect representations Valence-Arousal-Dominance (VAD) from the latent space. We also enhance the disentangled representations by introducing VAD supervision signals from a sentiment lexicon and minimising the mutual information between VAD distributions. Experiments show that VAD-VAE outperforms the state-of-the-art model on two datasets. Further analysis proves the effectiveness of each proposed module and the quality of disentangled VAD representations.
Dimensional sentiment analysis (DSA) aims to compute real-valued sentiment scores of texts in multiple dimensions such as valence and arousal. Existing methods for DSA are usually based on supervised learning. However...
详细信息
Dimensional sentiment analysis (DSA) aims to compute real-valued sentiment scores of texts in multiple dimensions such as valence and arousal. Existing methods for DSA are usually based on supervised learning. However, it is expensive and time-consuming to annotate sufficient samples for training. In this paper, we propose a semi-supervised approach for DSA based on the variational autoencoder model. Our model consists of three modules: an encoding module to encode sentences into hidden vectors, a sentiment prediction module to predict the sentiment scores of sentences, and a decoding module that takes the outputs of the preceding two modules as input and reconstructs the input sentences. In our approach, the sentiment prediction module is encouraged to accurately predict sentiment scores of both labeled and unlabeled texts to help the decoding module reconstruct such texts more accurately. Thus, our approach can exploit useful information in unlabeled data. Experimental results on three benchmark datasets show that our approach can effectively improve the performance of DSA with considerably less labeled data. (C) 2018 Elsevier B.V. All rights reserved.
For humans to trust in artificial intelligence (AI) systems, it is essential for machine learning (ML) models to be interpretable to users. For example, the judicial process requires that AI conclusions must be rigoro...
详细信息
For humans to trust in artificial intelligence (AI) systems, it is essential for machine learning (ML) models to be interpretable to users. For example, the judicial process requires that AI conclusions must be rigorous and absolutely interpretable. In this paper, we propose a novel approach, VAE-SLIME, for providing stable local interpretable model-agnostic explanations (SLIME) based on a variational autoencoder (VAE). LIME is a technique that explains the predictions of any classifier in an interpretable and faithful manner. Despite the great success of LIME, the most popular method in this category, it has several disadvantages due to its random perturbation-based sampling method. The VAE-SLIME proposed in this paper is specifically designed to address the lack of stability and local fidelity exhibited by LIME for tabular data. VAE-SLIME first employs fixed noise to replace the random Gaussian noise used by the reparameterization trick of the VAE. Then, it uses this new VAE model instead of random perturbation method to generate stable samples. By considering the sequential relationship and flipping of features, a novel explanation stability evaluation metric, the feature sequence stability index (FSSI), is introduced to accurately evaluate the stability of explanations. In a comparison with 6 state-of-the-art approaches on 7 commonly used tabular datasets, the experimental results show beyond doubt that the explanations produced by our approach are most stable, and its local fidelity is 65.17% higher than that of other approaches on average.
Collaborative filtering (CF) has been generally used in recommender systems when faced some practical problems. Due to the sparsity of the rating matrix, the traditional CF-based approach has a significant decline in ...
详细信息
Collaborative filtering (CF) has been generally used in recommender systems when faced some practical problems. Due to the sparsity of the rating matrix, the traditional CF-based approach has a significant decline in recommendation performance. Gradually, a hybrid method, using side information and rating information, has been widely employed and achieves great performance. Together with side information and rating information, the hybrid method can overcome the data sparsity and cold-start problems. However, they seem to fail to take into consideration the fact that the sparsity of single side information. To solve this problem, we take full advantage of the characteristics of deep learning that can learn effective representation and propose a novel deep learning model named additional variational autoencoder that considers both content and tag information of the item. The model learns effective latent representations from additional side information, including content information and tag information in an unsupervised manner. With the help of graphical models, it can extract the implicit relationships between users and items effectively. A large number of experimental results on two actual datasets show that our proposed model is superior to other methods, and the performance improvement is achieved.
Data-driven soft sensors have been widely used in industrial processes for over two decades. Industrial processes often exhibit nonlinear and time-varying behavior due to complex physical and chemical mechanisms, feed...
详细信息
Data-driven soft sensors have been widely used in industrial processes for over two decades. Industrial processes often exhibit nonlinear and time-varying behavior due to complex physical and chemical mechanisms, feedback control, and dynamic noise. Lately, variational autoencoder (VAE) has arisen as one of the most prevalent methods for unsupervised learning of intricate distributions. Despite being successful in deep feature extraction and uncertain data modeling, it still suffers from instability and reconstruction error due to random sampling in the latent subspace representation of original input space. In this article, to deal with those limitations, constrained VAE (CVAE) is proposed by utilizing input sample information. Enthused by parallel interaction mechanism between the ventral and dorsal stream of the human brain in object recognition, parallel interaction spatial-temporal CVAE (PIST-CVAE) is proposed to extract spatial and temporal features from input samples. Lower dimensional nonlinear features extracted from PIST-CVAE are used to build the soft sensor. The effectiveness of CVAE and PIST-CVAE is demonstrated in an industrial case study, a polyester polymerization process. The obtained results demonstrate that CVAE is able to reconstruct inputs with higher accuracy and the proposed PIST-CVAE-based soft sensor yields more accurate estimations for the melt viscosity index of the polymerization process.
There is a class-imbalance problem that the number of minority class samples is significantly lower than that of majority class samples in common network traffic datasets. Class-imbalance phenomenon will affect the pe...
详细信息
There is a class-imbalance problem that the number of minority class samples is significantly lower than that of majority class samples in common network traffic datasets. Class-imbalance phenomenon will affect the performance of the classifier and reduce the robustness of the classifier to detect unknown anomaly detection. And the distribution of the continuous features in the dataset does not follow the Gaussian distribution, which will bring great difficulties to intrusion detection. We propose Conditional Wasserstein variational autoencoders with Generative Adversarial Network (CWVAEGAN) to solve the class-imbalance phenomenon, CWVAEGAN transform the original dataset through data preprocessing, and then use the improved VAEGAN to generate minority class samples. According to the CWVAEGAN model, an intrusion detection system based on CWVAEGAN and One-dimensional convolutional neural networks (1DCNN), namely CWVAEGAN-1DCNN, is established. By using the examples generated by CWVAEGAN, the problem of intrusion detection on class unbalanced data is solved. Specifically, CWVAEGAN-1DCNN consists of three modules: data preprocessing module, CWVAEGAN, and deep neural network. We evaluate the performance of CWVAEGAN-1DCNN on two benchmark datasets and compared it with the other 16 methods. Experiment results suggest that the performance of CWVAEGAN-1DCNN is better than class-balancing methods, and other advanced methods.
We present an inversion algorithm with a deep-learning-based model compression scheme. Models are described with latent parameters of a trained variational autoencoder (VAE) neural network. Given observed data, latent...
详细信息
We present an inversion algorithm with a deep-learning-based model compression scheme. Models are described with latent parameters of a trained variational autoencoder (VAE) neural network. Given observed data, latent parameters are inverted by minimizing the data misfit cost function using the Gauss-Newton method. This inversion algorithm is tested using both synthetic and experimental datasets. We achieve a 0.87% compression rate while maintaining high-quality reconstruction. The deep neural network renders nonlinear model compression, which largely reduces the number of unknowns;hence, it has higher computational efficiency. Furthermore, various prior knowledge that is difficult to describe with rigorous forms can be incorporated into inversion through training the neural network, which mitigates the ill-posedness of the inverse problem.
We propose a novel approach based on variational autoencoder with Gaussian mixture latent space (GMVAE) to address the challenging problem of pulse shape discrimination (PSD) in organic scintillators, in the presence ...
详细信息
We propose a novel approach based on variational autoencoder with Gaussian mixture latent space (GMVAE) to address the challenging problem of pulse shape discrimination (PSD) in organic scintillators, in the presence of pile up. Unlike deterministic charge integration, which is very sensitive to pulse -processing parameters, the GMVAE performances are robust against variations of the hyperparameters. When compared to other supervised machine learning methods, GMVAE requires the fewest training pulses (100) to achieve a classification accuracy within 2% of its optimum performance, i.e. , 98.3% accuracy. GMVAE exhibited excellent classification despite the difference in energy spectra between the training and test data sets, which were 14.1 MeV neutron pulses and 239 PuBe pulses, respectively. While requiring minimum supervision, GMVAE showed superior PSD performances compared to both deterministic and supervised machine learning approaches. GMVAE is hence particularly suitable for real-time pulse classification, where expert labeling is unavailable and fine tuning of the discrimination parameters impractical.
暂无评论