检索结果-内蒙古大学图书馆

Prior latent distribution comparison for the RNN variational autoencoder in low-resource language modeling

JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022年第5期42卷 4541-4549页

作者： Kostiuk, Yevhen Lukashchuk, Mykola Gelbukh, Alexander Sidorov, Grigori Taras Shevchenko Natl Univ Kyiv Kiev Ukraine Inst Politecn Nacl Ctr Invest Comp Mexico City DF Mexico

Probabilistic Bayesian methods are widely used in the machine learning domain. variational autoencoder (VAE) is a common architecture for solving the Language Modeling task in a self-supervised way. VAE consists of a concept of latent variables inside the model. Latent variables are described as a random variable that is fit by the data. Up to now, in the majority of cases, latent variables are considered normally distributed. The normal distribution is a well-known distribution that can be easily included in any pipeline. Moreover, the normal distribution is a good choice when the Central Limit Theorem (CLT) holds. It makes it effective when one is working with i.i.d. (independent and identically distributed) random variables. However, the conditions of CLT in Natural Language Processing are not easy to check. So, the choice of distribution family is unclear in the domain. This paper studies the priors selection impact of continuous distributions in the Low-Resource Language Modeling task with VAE. The experiment shows that there is a statistical difference between the different priors in the encoder-decoder architecture. We showed that family distribution hyperparameter is important in the Low-Resource Language Modeling task and should be considered for the model training.

关键词： Bayesian model low-resource language modeling NLP priors RNN VAE variational autoencoder

来源：评论

学校读者我要写书评

暂无评论

Emotion-Guided Music Accompaniment Generation Based on variational autoencoder

Emotion-Guided Music Accompaniment Generation Based on Varia...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Wang, Qi Zhang, Shubing Zhou, Li China Univ Geosci Wuhan Peoples R China

ISBN: (纸本)9781665488679

Music accompaniment generation is a crucial aspect in the composition process. Deep neural networks have made significant strides in this field, but it remains a challenge for AI to effectively incorporate human emotions to create beautiful accompaniments. Existing models struggle to effectively characterize human emotions within neural network models while composing music. To address this issue, we propose the use of an easy-to-represent emotion flow model, the Valence/Arousal Curve, which allows for the compatibility of emotional information within the model through data transformation and enhances interpretability of emotional factors by utilizing a variational autoencoder as the model structure. Further, we used relative self-attention to maintain the structure of the music at music phrase level and to generate a richer accompaniment when combined with the rules of music theory. Our experimental results indicate that the emotional flow of the music generated by our model has a strong correlation with the input emotion, demonstrating the model's strong interpretability and control of emotional flow. The generated music is also well-structured, diverse, and dynamic, outperforming the baseline models.

关键词： Music Accompaniment Generation Emotional Flow variational autoencoder Rule constraints

来源：评论

学校读者我要写书评

暂无评论

FedVAE: Trajectory privacy preserving based on Federated variational autoencoder 98

FedVAE: Trajectory privacy preserving based on Federated Var...

引用

98th IEEE Vehicular Technology Conference (VTC-Fall)

作者： Jiang, Yuchen Wu, Ying Zhang, Shiyao Yu, James J. Q. Southern Univ Sci & Technol Dept Comp Sci & Engn Shenzhen Peoples R China Southern Univ Sci & Technol Res Inst Trustworthy Autonomous Syst Shenzhen Peoples R China Univ York Dept Comp Sci York N Yorkshire England

ISBN: (纸本)9798350329285

The use of trajectory data with abundant spatial-temporal information is pivotal in Intelligent Transport Systems (ITS) and various traffic system tasks. Location-Based Services (LBS) capitalize on this trajectory data to offer users personalized services tailored to their location information. However, this trajectory data contains sensitive information about users' movement patterns and habits, necessitating confidentiality and protection from unknown collectors. To address this challenge, privacy-preserving methods like K-anonymity and Differential Privacy have been proposed to safeguard private information in the dataset. Despite their effectiveness, these methods can impact the original features by introducing perturbations or generating unrealistic trajectory data, leading to suboptimal performance in downstream tasks. To overcome these limitations, we propose a Federated variational autoencoder (FedVAE) approach, which effectively generates a new trajectory dataset while preserving the confidentiality of private information and retaining the structure of the original features. In addition, FedVAE leverages variational autoencoder (VAE) to maintain the original feature space and generate new trajectory data, and incorporates Federated Learning (FL) during the training stage, ensuring that users' data remains locally stored to protect their personal information. The results demonstrate its superior performance compared to other existing methods, affirming FedVAE as a promising solution for enhancing data privacy and utility in location-based applications.

关键词： Federated learning privacy preserving trajectory generation variational autoencoder

来源：评论

学校读者我要写书评

暂无评论

Unsupervised Disentanglement Learning via Dirichlet variational autoencoder 36th

Unsupervised Disentanglement Learning via Dirichlet Variatio...

引用

36th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE)

作者： Xu, Kunxiong Fan, Wentao Liu, Xin Huaqiao Univ Dept Comp Sci & Technol Quanzhou Peoples R China Beijing Normal Univ Hong Kong Baptist Univ United Dept Comp Sci Zhuhai Peoples R China BNU HKBU United Int Coll Guangdong Prov Key Lab Interdisciplinary Res & Ap Zhuhai Peoples R China

ISBN: (纸本)9783031368189;9783031368196

Unsupervised disentanglement learning is the process of discovering factorized variables that include interpretable semantic information and encode separate factors of variations in the data. It is a critical learning problem and has been applied in various tasks and domains. Most of the existing unsupervised disentanglement learning methods are based on the variational autoencoder (VAE) and adopt Gaussian distribution as the prior over the latent space. However, these methods suffer from a collapse of the decoder weights, which leads to degraded disentangling ability, due to the Gaussian prior. To address this issue, in this paper we propose a novel unsupervised disentanglement learning method based on a VAE framework in which the Dirichlet distribution is deployed as the prior over latent space. In our method, the interpretable factorised latent representations can be obtained by balancing the capacity of the latent information channel and the learning of statistically independent latent factors. The effectiveness of our method is validated through experiments on several publicly available datasets.

关键词： Disentanglement learning variational autoencoder Dirichlet distribution

来源：评论

学校读者我要写书评

暂无评论

A Study on Developmental Artificial Neural Networks that Integrate Multiple Functions using variational autoencoder 10

A Study on Developmental Artificial Neural Networks that Int...

引用

10th International Conference on Soft Computing and Machine Intelligence (ISCMI)

作者： Iwai, Haruka Kobayashi, Ichiro Ochanomizu Univ Grad Sch Humanities & Sci Adv Sci Tokyo Japan

ISBN: (纸本)9798350359374

In conventional deep learning fine tuning or transfer learning, the synaptic weights of neural networks that have been previously learned and acquired in a particular task solution are transferred to the learning of a similar task to achieve efficient learning of that task. In this case, both tasks are assumed to be similar, and the function to be transferred to solve a task is a single one. On the other hand, organisms can abstract multiple experiences acquired in the past and adapt them as knowledge to new tasks. This requires a new neural network architecture that fuses neural networks that have learned individual problem-solving functions and is capable of transfer learning of multiple functions. To acquire such an architecture, we aim to develop a framework to fuse neural networks with multiple individual functions to acquire a new network topology. Based on this, in this study, we develop a Developmental Artificial Neural Networks (DANN) capable of simultaneously transferring multiple functions that generate a neural network with new functions that integrate multiple functions. we use the framework of Weight Agnostic Neural Networks (WANN), in which the topology is acquired evolutionarily rather than by the synaptic weight of the networks, and the obtained topology is transformed into a matrix by using an embedded representation of the generation rules, which is then used as input to the encoder-decoder model based on Grammar variational autoencoder (GVAE). Through the experimental results, it was confirmed that multiple neural network models that solve individual tasks can be represented in the latent space, and that capturing the relationships among tasks.

关键词： AutoML Transfer Learning Weight Agnostic Neural Networks variational autoencoder

来源：评论

学校读者我要写书评

暂无评论

A Disentangled Recurrent variational autoencoder for Speech Enhancement

A Disentangled Recurrent Variational Autoencoder for Speech ...

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yan, Hegen Lu, Zhihua Ningbo Univ Coll Informat Sci & Engn Ningbo Peoples R China

ISBN: (纸本)9781665468916

The variational autoencoder (VAE) is a powerful generative model of speech signal. In the problem of speech enhancement, the VAE is used to model the clean speech whereas the non-negative matrix factorization is used to model the noise. The Wiener filter can be applied to extract the clean speech from the mixture signal. In this paper, we extend this approach by proposing a more sophisticated VAE framework, so-called disentangled recurrent variational autoencoder (RVAE), to model the clean speech. It takes the strategy of pre-training plus finetuning. Firstly, the clean speech and noise are pre-trained using two dependent RVAEs. Secondly, by transferring the knowledge learned in these two RVAEs, the third RVAE is used to train the mixture signal, resulting in disentanglement of the latent variables of the clean speech and noise. Experimental results demonstrate that the proposed approach outperforms other VAE-based approaches in terms of the signal-to-distortion ratio.

关键词： speech enhancement variational autoencoder disentanglement transfer learning supervised training

来源：评论

学校读者我要写书评

暂无评论

A Physically Constrained variational autoencoder for Geochemical Pattern Recognition

引用

MATHEMATICAL GEOSCIENCES 2022年第4期54卷 783-806页

作者： Xiong, Yihui Zuo, Renguang Luo, Zijing Wang, Xueqiu China Univ Geosci State Key Lab Geol Proc & Mineral Resources Wuhan 430074 Peoples R China Minist Nat Resources Key Lab Geochem Explorat Inst Geophys & Geochem Explorat Langfang 065000 Peoples R China UNESCO Int Ctr Global Scale Geochem Langfang 065000 Peoples R China

Quantification and recognition of geochemical patterns are extremely important for geochemical prospecting and can facilitate a better understanding of regional metallogenesis. Recognition of such patterns with deep learning (DL) algorithms has attracted considerable attention, as these algorithms can generally extract high-level geochemical features and thus create models in which geochemical patterns are fully exploited. These DL algorithms are usually constructed to be compatible and consistent with the underlying data, but their interpretability and pertinent physical constraints (such as granitic intrusions related to magmatic-hydrothermal mineralization) are generally overlooked. This paper introduces a physically constrained variational autoencoder (VAE) architecture to identify geochemical patterns associated with tungsten polymetallic mineralization. We first identify physical constraints from geological characteristics and metallogenic regulation via the methods of fry analysis, standard deviation ellipses, and fractal analysis to reveal the controlling function of granitic intrusions on mineralization. Subsequently, we construct the physical constraints based on the nonlinear controlling function and add the geological constraints into the VAE loss function as a penalty term. After optimization of the network parameters, the well-trained physically constrained VAE architecture can recognize the geochemical anomaly patterns that the conventional VAE cannot. These extracted geochemical anomaly patterns generally show a strong spatial relationship with the granitic intrusions. In addition, the performance measures involving the receiver operating characteristic curve and success-rate further indicate that the generalization accuracy of the conventional VAE can be enhanced via physics-based regularization. These results suggest that the proposed physically constrained VAE, which integrates physical knowledge into the VAE loss function, not only improves the g

关键词： Geochemical patterns Physically constrained deep learning variational autoencoder Geochemical exploration

来源：评论

学校读者我要写书评

暂无评论

Predicting customer churn for platform businesses: using latent variables of variational autoencoder as consumers' purchasing behavior

引用

NEURAL COMPUTING & APPLICATIONS 2022年第21期34卷 18525-18541页

作者： Hasumoto, Kyosuke Goto, Masayuki Waseda Univ Inst Data Sci Tokyo Japan Waseda Univ Fac Sci & Engn Tokyo Japan

Customer churn is considered a critical issue for all businesses, as customer loss leads to a decrease in future profits. Although acquiring new customers can address such losses, this process tends to cost more than retaining existing customers. Therefore, identifying potential churners and then retaining them is important. While churn prediction has been studied widely, current research must analyze more complex business models in response to their increase, such as platform businesses. However, modeling churn prediction for these businesses is challenging because consumer behavior over platforms is more complicated. Past approaches to churn prediction can be improved upon using recent advancements in deep learning that capture the nonlinear relationships behind the data in a data-driven manner, especially for complex business models. This study proposes a method of extracting latent features from purchase histories as explanatory variables for churn prediction using a variational autoencoder with the actual customer distribution as a prior. The proposed method is validated using real purchase data from a platform business and shows a 1.5% improvement in F-measure against the baseline, and a 20% improvement for customers with recent transactions. Subsequently, the variables are examined using several methods for data analysis to interpret the meanings of the extracted features and underlying customers' purchasing behavior in the group of potential churners. With these analyses, the model provides practical implications for understanding what kind of purchasing behavior may lead to churn and planning effective retention strategies.

关键词： variational autoencoder Deep learning Churn prediction RFM Customer lifetime value Consumer purchase behavior

来源：评论

学校读者我要写书评

暂无评论

Enhancing Robot Self-Localization Accuracy and Robustness: A variational autoencoder-Based Approach 5

Enhancing Robot Self-Localization Accuracy and Robustness: A...

引用

5th International Conference on Control and Robotics (ICCR)

作者： Abe, Asuma Pongthanisorn, Goragod Kaneko, Shin-ichiro Capi, Genci Hosei Univ Dept Mech Engn Grad Sch Sci & Engn Tokyo Japan Natl Inst Technol Toyama Coll Dept Elect & Control Syst Engn Toyama Japan

ISBN: (纸本)9798350307627

Self-localization is a crucial task for robots, demanding high accuracy. In this work, we propose a new robot localization method based on the variational autoencoder (VAE). In our method, the robot utilizes the captured image to generate robot localization in indoor environments. The utilization of VAE makes the system adaptive to varying environmental conditions. Our findings demonstrate that utilizing both the robot's coordinates and images as training data significantly enhances the accuracy of robot self-localization estimation and improves the robustness of the system due to sensor data noise.

关键词： robot localization variational autoencoder wheelchair robot indoor environments

来源：评论

学校读者我要写书评

暂无评论

VAE-AD: Unsupervised variational autoencoder for Anomaly Detection in Hyperspectral Images 29th

VAE-AD: Unsupervised Variational Autoencoder for Anomaly Det...

引用

29th International Conference on Neural Information Processing

作者： Ojha, Nikhil Sinha, Indrajeet Kumar Singh, Krishna Pratap Indian Inst Informat Technol Allahabad Machine Learning & Optimizat Lab Dept Informat Technol Prayagraj UP India

ISBN: (纸本)9789819916474;9789819916481

Anomaly detection in hyperspectral images is an important and challenging problem. Most available data sets are unlabeled, and very few are labelled. In this paper, we proposed a lightweight variational autoencoder anomaly detector (VAE-AD) for hyperspectral data. VAE is used to learn the background distribution of the image, and thereafter it is used to construct a background representation for each pixel. Further reconstruction error is calculated between the background reconstructed image and the original image used for anomaly detection. A GMM-based post-processing step is used to construct the final detection map. The comparative analysis with five real-world hyperspectral data sets shows that the proposed model achieves better or comparable results with few learning parameters of the model, and with less time.

关键词： Anomaly detection variational autoencoder HSI hyperspectral imaging

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：