We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still...
详细信息
ISBN:
(纸本)9781450356398
We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research. We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm has information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.
The task of epileptic focus localization receives great attention due to its role in an effective epileptic surgery. The clinicians highly depend on the intracranial EEG data to make a surgical decision related to epi...
详细信息
The task of epileptic focus localization receives great attention due to its role in an effective epileptic surgery. The clinicians highly depend on the intracranial EEG data to make a surgical decision related to epileptic subjects suffering from uncontrollable seizures. This surgery usually aims to remove the epileptogenic region which requires precise characterization of that area using the EEG recordings. In this paper, we propose two methods based on deep learning targeting accurate automatic epileptic focus localization using the non-stationary EEG recordings. Our first proposed method is based on semi-supervised learning, in which a deep convolutional autoencoder is trained and then the pre-trained encoder is used with multi-layer perceptron as a classifier. The goal is to determine the location of the EEG signal that is responsible for the epileptic activity. In the second proposed method, unsupervised learning scheme is implemented by merging deep convolutional variational autoencoder and K-means algorithm for clustering the iEEG signals into two distinct clusters based on the seizure source. The proposed methods automate and integrate the features extraction and classification processes instead of manually extracting the features as done in the previous studies. Dimensionality reduction is achieved using the autoencoder, while the important spatio-temporal features are extracted from the EEG recordings using the convolutional layers. Moreover, we implemented the inference network of the semi-supervised model on FPGA. The results of our experiments demonstrate high classification accuracy and clustering performance in localizing the epileptic focus compared with the state of the art.
Creating realistic animations of human faces is still a challenging task in computer graphics. While computer graphics (CG) models capture much variability in a small parameter vector, they usually do not meet the nec...
详细信息
Creating realistic animations of human faces is still a challenging task in computer graphics. While computer graphics (CG) models capture much variability in a small parameter vector, they usually do not meet the necessary visual quality. This is due to the fact, that geometry-based animation often does not allow fine-grained deformations and fails in difficult areas (mouth, eyes) to produce realistic renderings. Image-based animation techniques avoid these problems by using dynamic textures that capture details and small movements that are not explained by geometry. This comes at the cost of high-memory requirements and limited flexibility in terms of animation because dynamic texture sequences need to be concatenated seamlessly, which is not always possible and prone to visual artefacts. In this study, the authors present a new hybrid animation framework that exploits recent advances in deep learning to provide an interactive animation engine that can be used via a simple and intuitive visualisation for facial expression editing. The authors describe an automatic pipeline to generate training sequences that consist of dynamic textures plus sequences of consistent three-dimensional face models. Based on this data, they train a variational autoencoder to learn a low-dimensional latent space of facial expressions that is used for interactive facial animation.
Background and Aims: Annually, 4 million people are affected by ulcers in the GI tract, and poorly managed ulcers can lead to adverse events that may increase the risk of developing fatal diseases such as gastric canc...
详细信息
Background and Aims: Annually, 4 million people are affected by ulcers in the GI tract, and poorly managed ulcers can lead to adverse events that may increase the risk of developing fatal diseases such as gastric cancer, GI bleeding, ulcerative colitis, and other conditions. Deep learning–based computer-aided diagnostics is essential for automating the diagnosis of such illnesses. It aids clinicians in diagnosis by reducing the time of diagnosis and lowering diagnostic errors. Data shortage is anticipated in endoscopy settings, and small data sets prevent the performance of the deep learning model from being generalized. The current study addresses data scarcity by expanding the available data through the artificial generation of new data. Methods: This study exploits the combination of the generative adversarial network (GAN) and variational autoencoders (VAE) and develops a VAE-GAN architecture to generate artificial endoscopic images. VAE-GAN offers resilience toward mode collapse, vanishing gradient, instability, and non-convergence. The generated images were tested against the trained DenseNet121 model with a higher classification threshold for ulcer identification and estimated precision and recall scores that capture the quality and diversity of the generated images, respectively. Furthermore, the generated images were also presented and compared with the clinical interpretation by an expert. The effectiveness of data augmentation by VAE-GAN was also evaluated with the developed 5-layer convolutional neural network (CNN) model for ulcer classification. Results: The proposed VAE-GAN architecture synthesized artificial realistic endoscopic images. With the trained DenseNet121 model under a higher threshold for ulcer detection, the generated images of ulcers achieved 99% precision and 92% recall. The expert achieved 57.1% classification accuracy for artificial versus real ulcer examination. The 5-layer CNN model achieved 94% classification accuracy on the test d
Building a comprehensive topic model has become an important research tool in single-cell genomics. With a topic model, we can decompose and ascertain distinctive cell topics shared across multiple cells, and the gene...
详细信息
Building a comprehensive topic model has become an important research tool in single-cell genomics. With a topic model, we can decompose and ascertain distinctive cell topics shared across multiple cells, and the gene programs implicated by each topic can later serve as a predictive model in translational studies. Here, we present a Bayesian topic model that can uncover short-term RNA velocity patterns from a plethora of spliced and unspliced single-cell RNA-sequencing (RNA-seq) counts. We showed that modeling both types of RNA counts can improve robustness in statistical estimation and can reveal new aspects of dynamic changes that can be missed in static analysis. We showcase that our modeling framework can be used to identify statistically significant dynamic gene programs in pancreatic cancer data. Our results discovered that seven dynamic gene programs (topics) are highly correlated with cancer prognosis and generally enrich immune cell types and pathways.
Different spatial point process models and techniques have been developed in the past decades to facilitate the statistical analysis of spatial point patterns. However, in some cases the spatial point process methodol...
详细信息
Different spatial point process models and techniques have been developed in the past decades to facilitate the statistical analysis of spatial point patterns. However, in some cases the spatial point process methodology is scarce and no flexible models nor suitable statistical methods are available. For example, due to its complexity, the statistical analysis of spatial point patterns of several groups observed at a number of time instances has not been studied in-depth, and there are a few limited models and methods available for such data. In the present work, we provide a mathematical framework for coupling neural network methods with the statistical analysis of point patterns. In particular, we discuss an example of deep neural networks in the statistical analysis of highly multivariate spatial point patterns and provide a new strategy for building spatio-temporal point processes using variational autoencoder generative neural networks. (c) 2022 Elsevier B.V. All rights reserved.
The COVID-19 pandemic has highlighted the importance of non-pharmacological interventions (NPIs) for controlling epidemics of emerging infectious diseases. Despite their importance, NPIs have been monitored mainly thr...
详细信息
The COVID-19 pandemic has highlighted the importance of non-pharmacological interventions (NPIs) for controlling epidemics of emerging infectious diseases. Despite their importance, NPIs have been monitored mainly through the manual efforts of volunteers. This approach hinders measurement of the NPI effectiveness and development of evidence to guide their use to control the global pandemic. We present EpiTopics, a machine learning approach to support automation of NPI prediction and monitoring at both the document level and country level by mining the vast amount of unlabeled news reports on COVID-19. EpiTopics uses a 3-stage, transfer-learning algorithm to classify documents according to NPI categories, relying on topic modeling to support result interpretation. We identified 25 interpretable topics under 4 distinct and coherent COVID-related themes. Importantly, the use of these topics resulted in significant improvements over alternative automated methods in predicting the NPIs in labeled documents and in predicting country-level NPIs for 42 countries.
Recent years have seen a surge in interest in building deep learning-based fully data-driven models for weather prediction. Such deep learning models, if trained on observations can mitigate certain biases in current ...
详细信息
Recent years have seen a surge in interest in building deep learning-based fully data-driven models for weather prediction. Such deep learning models, if trained on observations can mitigate certain biases in current state-of-the-art weather models, some of which stem from inaccurate representation of subgrid-scale processes. However, these data-driven models, being over-parameterized, require a lot of training data which may not be available from reanalysis (observational data) products. Moreover, an accurate, noise-free, initial condition to start forecasting with a data-driven weather model is not available in realistic scenarios. Finally, deterministic data-driven forecasting models suffer from issues with long-term stability and unphysical climate drift, which makes these data-driven models unsuitable for computing climate statistics. Given these challenges, previous studies have tried to pre-train deep learning-based weather forecasting models on a large amount of imperfect long-term climate model simulations and then re-train them on available observational data. In this article, we propose a convolutional variational autoencoder (VAE)-based stochastic data-driven model that is pre-trained on an imperfect climate model simulation from a two-layer quasi-geostrophic flow and re-trained, using transfer learning, on a small number of noisy observations from a perfect simulation. This re-trained model then performs stochastic forecasting with a noisy initial condition sampled from the perfect simulation. We show that our ensemble-based stochastic data-driven model outperforms a baseline deterministic encoder-decoder-based convolutional model in terms of short-term skills, while remaining stable for long-term climate simulations yielding accurate climatology. Impact Statement A stochastic VAE-based data-driven model pre-trained on imperfect climate simulations and re-trained with transfer learning, on a limited number of observations, leads to accurate short-term w
In manufacturing industries, monitoring the complicated devices often necessitates automated methods that can leverage the multivariate time series data produced by the machines. However, analyzing this data can be ch...
详细信息
In manufacturing industries, monitoring the complicated devices often necessitates automated methods that can leverage the multivariate time series data produced by the machines. However, analyzing this data can be challenging due to varying noise levels in the data and possible nonlinear relations between the process variables, requiring appropriate tools to deal with such properties.
This thesis proposes a deep learning-based approach to detect anomalies and interpret their root causes from multivariate time series data, which can be applied in a near real-time setting. The proposed approach extends an existing model from the literature, which employs a variational autoencoder architecture and recurrent neural networks to capture both stochasticity and temporal relations of the data.
The anomaly detection and root cause interpretation performance of the proposed method is compared against five baseline algorithms previously proposed in the literature using real-world data collected from plastic injection molding machines and artificially generated multivariate time series data.
The results of this thesis show that the proposed method performs well on the evaluated multivariate time series datasets, mostly outperforming the baseline methods. Additionally, the approach had the best performance among the selected methods on providing root cause interpretation of the detected anomalies. The experiments conducted in this thesis suggest that deep learning-based algorithms are beneficial for anomaly detection in scenarios where the problem is too complicated for traditional methods, and enough training data is available. However, the amount of real-world injection molding machine data used in the experiments is relatively small, and therefore further experiments should be performed with larger datasets to obtain more generalizable results.
Background: One of the main concerns of public health surveillance is to preserve the physical and mental health of older adults while supporting their independence and privacy. On the other hand, to better assist tho...
详细信息
Background: One of the main concerns of public health surveillance is to preserve the physical and mental health of older adults while supporting their independence and privacy. On the other hand, to better assist those individuals with essential health care services in the event of an emergency, their regular activities should be monitored. Internet of Things (IoT) sensors may be employed to track the sequence of activities of individuals via ambient sensors, providing real-time insights on daily activity patterns and easy access to the data through the connected ecosystem. Previous surveys to identify the regular activity patterns of older adults were deficient in the limited number of participants, short period of activity tracking, and high reliance on predefined normal activity. Objective: The objective of this study was to overcome the aforementioned challenges by performing a pilot study to evaluate the utilization of large-scale data from smart home thermostats that collect the motion status of individuals for every 5-minute interval over a long period of time. Methods: From a large-scale dataset, we selected a group of 30 households who met the inclusion criteria (having at least 8 sensors, being connected to the system for at least 355 days in 2018, and having up to 4 occupants). The indoor activity patterns were captured through motion sensors. We used the unsupervised, time-based, deep neural-network architecture long short-term memory-variational autoencoder to identify the regular activity pattern for each household on 2 time scales: annual and weekday. The results were validated using 2019 records. The area under the curve as well as loss in 2018 were compatible with the 2019 schedule. Daily abnormal behaviors were identified based on deviation from the regular activity model. Results: The utilization of this approach not only enabled us to identify the regular activity pattern for each household but also provided other insights by assessing sleep beh
暂无评论