The biomedical literature provides a rich source of knowledge such as protein-protein interactions (PPIs), drug-drug interactions (DDIs) and chemical-protein interactions (CPIs). Biomedical relation extraction aims to...
详细信息
The biomedical literature provides a rich source of knowledge such as protein-protein interactions (PPIs), drug-drug interactions (DDIs) and chemical-protein interactions (CPIs). Biomedical relation extraction aims to automatically extract biomedical relations from biomedical text for various biomedical research. State-of-the-art methods for biomedical relation extraction are primarily based on supervised machine learning and therefore depend on (sufficient) labeled data. However, creating large sets of training data is prohibitively expensive and labor-intensive, especially so in biomedicine as domain knowledge is required. In contrast, there is a large amount of unlabeled biomedical text available in PubMed. Hence, computational methods capable of employing unlabeled data to reduce the burden of manual annotation are of particular interest in biomedical relation extraction. We present a novel semi-supervised approach based on variational autoencoder (VAE) for biomedical relation extraction. Our model consists of the following three parts, a classifier, an encoder and a decoder. The classifier is implemented using multi-layer convolutional neural networks (CNNs), and the encoder and decoder are implemented using both bidirectional long short-term memory networks (Bi-LSTMs) and CNNs, respectively. The semi-supervised mechanism allows our model to learn features from both the labeled and unlabeled data. We evaluate our method on multiple public PPI, DDI and CPI corpora. Experimental results show that our method effectively exploits the unlabeled data to improve the performance and reduce the dependence on labeled data. To our best knowledge, this is the first semi-supervised VAE-based method for (biomedical) relation extraction. Our results suggest that exploiting such unlabeled data can be greatly beneficial to improved performance in various biomedical relation extraction, especially when only limited labeled data (e.g. 2000 samples or less) is available in such ta
In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the dow...
详细信息
In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variational autoencoder (VAE). The VAE is proposed to learn independent low dimension representation while facing the problem that sometimes pre-set factors are ignored. We argue that the mutual information of the input and each learned factor of the representation plays a necessary indicator of discovering the influential factors. We find the VAE objective inclines to induce mutual information sparsity in factor dimension over the data intrinsic dimension and therefore result in some non-influential factors whose function on data reconstruction could be ignored. We show mutual information also influences the lower bound of VAE's reconstruction error and downstream classification task. To make such indicator applicable, we design an algorithm for calculating the mutual information for VAE and prove its consistency. Experimental results on MNIST, CelebA and DEAP datasets show that mutual information can help determine influential factors, of which some are interpretable and can be used to further generation and classification tasks, and help discover the variant that connects with emotion on DEAP dataset. (C) 2019 Elsevier Ltd. All rights reserved.
Semantic hashing is a technique to represent high-dimensional data using similarity-preserving binary codes for efficient indexing and search. Recently, variational autoencoders with Bernoulli latent representations a...
详细信息
ISBN:
(纸本)9783030934194;9783030934200
Semantic hashing is a technique to represent high-dimensional data using similarity-preserving binary codes for efficient indexing and search. Recently, variational autoencoders with Bernoulli latent representations achieved remarkable success in learning such codes in supervised and unsupervised scenarios, outperforming traditional methods thanks to their ability to handle the binary constraints architecturally. In this paper, we propose a novel method for supervision (self-supervised) of variational autoencoders where the model uses its own predictions of the label distribution to implement the pairwise objective function. Also, we investigate the robustness of hashing methods based on variational autoencoders to the lack of supervision, focusing on two semi-supervised approaches currently in use. Our experiments on text and image retrieval tasks show that, as expected, both methods can significantly increase the quality of the hash codes as the number of labelled observations increases, but deteriorates when the amount of labelled samples decreases. In this scenario, the proposed self-supervised approach outperforms the classical approaches and yields similar performance in fully-supervised settings.
Accurate 1-day global total electron content (TEC) forecasting is essential for ionospheric monitoring and satellite communications. However, it faces challenges due to limited data and difficulty in modeling long-ter...
详细信息
Accurate 1-day global total electron content (TEC) forecasting is essential for ionospheric monitoring and satellite communications. However, it faces challenges due to limited data and difficulty in modeling long-term dependencies. This study develops a highly accurate model for 1-day global TEC forecasting. We utilized generative TEC data augmentation based on the International Global Navigation Satellite Service (IGS) data set from 1998 to 2017 to enhance the model's prediction ability. Our model takes the TEC sequence of the previous 2 days as input and predicts the global TEC value for each hourly step of the next day. We compared the performance of our model with 1-day predicted ionospheric products provided by both the Center for Orbit Determination in Europe (C1PG) and Beihang University (B1PG). We proposed a two-step framework: (a) a time series generative model to produce realistic synthetic TEC data for training, and (b) an auto-correlation-based transformer model designed to capture long-range dependencies in the TEC sequence. Experiments demonstrate that our model significantly improves 1-day forecast accuracy over prior approaches. On the 2018 benchmark data set, the global root mean squared error (RMSE) of our model is reduced to 1.17 TEC units (TECU), while the RMSE of the C1PG model is 2.07 TECU. Reliability is higher in middle and high latitudes but lower in low latitudes (RMSE < 2.5 TECU), indicating room for improvement. This study highlights the potential of using data augmentation and auto-correlation-based transformer models trained on synthetic data to achieve high-quality 1-day global TEC forecasting.
The significance of customer churn analysis has escalated due to the increasing availability of relevant data and intensifying competition. Researchers and practitioners are focused on enhancing prediction accuracy in...
详细信息
The significance of customer churn analysis has escalated due to the increasing availability of relevant data and intensifying competition. Researchers and practitioners are focused on enhancing prediction accuracy in modeling approaches, with deep neural networks emerging as appealing due to their robust performance across domains. However, the computational demands surge due to the challenges posed by dimensionality and inherent characteristics of the data. To address these issues, this research proposes a novel hybrid model that strategically integrates Convolutional Neural Networks (CNN) and a modified variational autoencoder (VAE). By carefully adjusting the parameters of the VAE to capture the central tendency and range of variation, the study aims to enhance the effectiveness of classifying high-dimensional churn data. The proposed framework's efficacy is evaluated using six benchmark datasets from various domains, with performance metrics encompassing accuracy, f1-score, precision, recall, and response time. Experimental results underscore the prowess of the hybrid technique in effectively handling high-dimensional and imbalanced time series data, thus offering a robust pathway for enhanced churn analysis.
Reinforcement learning is a broad scheme of learning algorithms that, in recent times, has shown astonishing performance in controlling agents in environments presented as Markov decision processes. There are several ...
详细信息
Reinforcement learning is a broad scheme of learning algorithms that, in recent times, has shown astonishing performance in controlling agents in environments presented as Markov decision processes. There are several unsolved problems in current state-of-the-art that causes algorithms to learn suboptimal policies, or even diverge and collapse completely. Parts of the solution to address these issues may be related to short- and long-term planning, memory management and exploration for reinforcement learning algorithms. Games are frequently used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible and easy to control environments. Regardless, few games feature the ability to perceive how the algorithm performs exploration, memorization and planning. This article presents The Dreaming variational autoencoder with Stochastic Weight Averaging and Generative Adversarial Networks (DVAE-SWAGAN), a neural network-based generative modelling architecture for exploration in environments with sparse feedback. We present deep maze, a novel and flexible maze game-engine that challenges DVAE-SWAGAN in partial and fully observable state-spaces, long-horizon tasks and deterministic and stochastic problems. We show results between different variants of the algorithm and encourage future study in reinforcement learning driven by generative exploration.
In this work, we address the problem of targeted Value Added Tax (VAT) audit case selection by means of machine learning. This is a challenging problem that has remained rather elusive for EU-based Tax Departments, du...
详细信息
In this work, we address the problem of targeted Value Added Tax (VAT) audit case selection by means of machine learning. This is a challenging problem that has remained rather elusive for EU-based Tax Departments, due to the inadequate quantity of tax audits that can be used for conventional supervised model training. To this end, we devise a novel Gated Mixture variational autoencoder deep network, that can be effectively trained with data from a limited number of audited taxpayers, combined with a large corpus of filed VAT returns. This gives rise to a semi-supervised learning framework that leverages the latest advances in deep learning and robust regularization using variational inference. We developed our approach in collaboration with the Cyprus Tax Department and experimentally deployed it to facilitate its audit selection process;to this end, we used actual VAT data from Cyprus-based taxpayers. This way, we obtained strong empirical evidence that our approach can greatly facilitate the VAT audit case selection process. Specifically, we obtained up to 76% out-of-sample accuracy in detecting whether a significant tax yield will be generated from a specific prospective VAT audit. (C) 2019 Elsevier B.V. All rights reserved.
Recommendation in the modern world is not only about capturing the interaction between users and items, but also about understanding the relationship between items. Besides improving the quality of recommendation, it ...
详细信息
ISBN:
(纸本)9781450359405
Recommendation in the modern world is not only about capturing the interaction between users and items, but also about understanding the relationship between items. Besides improving the quality of recommendation, it enables the generation of candidate items that can serve as substitutes and supplements of another item. For example, when recommending Xbox, PS4 could be a logical substitute and the supplements could be items such as game controllers, surround system, and travel case. Therefore, given a network of items, our objective is to learn their content features such that they explain the relationship between items in terms of substitutes and supplements. To achieve this, we propose a generative deep learning model that links two variational autoencoders using a connector neural network to create Linked variational autoencoder (LVA). LVA learns the latent features of items by conditioning on the observed relationship between items. Using a rigorous series of experiments, we show that LVA significantly outperforms other representative and state-of-the-art baseline methods in terms of prediction accuracy. We then extend LVA by incorporating collaborative filtering (CF) to create CLVA that captures the implicit relationship between users and items. By comparing CLVA with LVA we show that inducing CF-based features greatly improve the recommendation quality of substitutable and supplementary items on a user level.
Generalized zero-shot learning (GZSL) aims to solve the category recognition tasks for unseen categories under the setting that training samples only contain seen classes while unseen classes are not available. This r...
详细信息
Generalized zero-shot learning (GZSL) aims to solve the category recognition tasks for unseen categories under the setting that training samples only contain seen classes while unseen classes are not available. This research is vital as there are always existing new categories and large amounts of unlabeled data in realistic scenarios. Previous work for GZSL usually maps the visual information of the visible classes and the semantic description of the invisible classes into the identical embedding space to bridge the gap between the disjointed visible and invisible classes, while ignoring the intrinsic features of visual images, which are sufficiently discriminative to classify themselves. To better use discriminative information from visual classes for GZSL, we propose the n-CADA-VAE. In our approach, we map the visual feature of seen classes to a high-dimensional distribution while mapping the semantic description of unseen classes to a low-dimensional distribution under the same latent embedding space, thus projecting information of different modalities to corresponding space positions more accurately. We conducted extensive experiments on four benchmark datasets (CUB, SUN, AWA1, and AWA2). The results show our model's superior performance in generalized zero-shot as well as few-shot learning.
Image interpolation is often implemented using one of two methods: optical flow or convolutional neural networks. These methods are typically pixel-based;they do not work well on objects between images far apart. Beca...
详细信息
Image interpolation is often implemented using one of two methods: optical flow or convolutional neural networks. These methods are typically pixel-based;they do not work well on objects between images far apart. Because they either rely on a simple frame average or pixel motion, they do not have the required knowledge of the semantic structure of the data. In this paper, we propose a method for image interpolation based on latent representations. We use a simple network structure based on a variational autoencoder and an adjustable hyperparameter that imposes the latent space distribution to generate accurate interpolation. To visualize the effects of the proposed approach, we evaluate a synthetic dataset. We demonstrate that our method outperforms both pixel-based methods and a conventional variational autoencoder, with particular improvements in nonsuccessive images.
暂无评论