Machine learning has shown remarkable artistic values and commercial potentials in the music industry. Recurrent variational autoencoders (RVAEs) have been widely applied to this area due to the condensing, inclusive,...
详细信息
ISBN:
(纸本)9783031013331;9783031013324
Machine learning has shown remarkable artistic values and commercial potentials in the music industry. Recurrent variational autoencoders (RVAEs) have been widely applied to this area due to the condensing, inclusive, and smooth nature of their latent space. However, RNNs are powerful auto-regressive models on their own, where the decoder in a RVAE can be strong enough to work independently from the encoder. When this happens, the model degrades from an autoencoder to a traditional RNN, which is known as posterior collapse. In this paper, we propose a cost-effective bar-wise regulation schema called MuseBar to alleviate this problem for music generation. We impose a prior on the hidden state of every music bar in the RNN encoder, instead of only on the last hidden state as in the standard RVAEs, such that the latent code is learned under stronger regulations. We further evaluate our proposed method, quantitatively and qualitatively, with extensive experiments on manually scraped musical data. The results demonstrate that the bar-wise regulation significantly improves the quality of the latent space in terms of Mutual Information and Kullback-Leibler divergence.
Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some invariance into representation. However...
详细信息
ISBN:
(纸本)9781450393850
Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some invariance into representation. However, adversarial learning methods are known to suffer from relatively unstable training, and this might harm the balance between fairness and predictiveness of representation. We propose a new approach, learning FAir Representation via distributional CONtrastive variational autoencoder (FarconVAE), which induces the latent space to be disentangled into sensitive and nonsensitive parts. We first construct the pair of observations with different sensitive attributes but with the same labels. Then, FarconVAE enforces each non-sensitive latent to be closer, while sensitive latents to be far from each other and also far from the non-sensitive latent by contrasting their distributions. We provide a new type of contrastive loss motivated by Gaussian and Student-t kernels for distributional contrastive learning with theoretical analysis. Besides, we adopt a new swap-reconstruction loss to boost the disentanglement further. FarconVAE shows superior performance on fairness, pretrained model debiasing, and domain generalization tasks from various modalities, including tabular, image, and text.
The mechanical behavior of composite interface can be influenced by multiple factors, including the morphological roughness, the structure of coating interphase, and the temperature. Here, high-throughput molecular dy...
详细信息
The mechanical behavior of composite interface can be influenced by multiple factors, including the morphological roughness, the structure of coating interphase, and the temperature. Here, high-throughput molecular dynamics (MD) simulations are carried out to investigate the entangled effects of these factors on the shear stiffness G, the friction coefficient mu, the debonding strain is an element of(d) and stress T-d, of SiCf/SiC interface. We find that G is maximized by small roughness and high temperature for the optimal chemical bonding effect;mu and.d are maximized by large roughness and low temperature, taking advantage of the mechanical interlocking effect while avoiding cusp softening;T-d demonstrates two local maxima which result from the competition between chemical bonding and mechanical interlocking. Provided the MD simulation results, a variational autoencoder (VAE) model is proposed to design the microstructure of SiCf/SiC interface for desired shear properties. According to the validations, the VAE-predicted interfacial configuration demonstrates highly similar shear properties to the reference one, justifying its potential for the microstructure design of composite interface. The results of this work can be employed to facilitate the development of SiCf/SiC composite by taking advantage of the synergistic effects of multiple designable factors.
This article introduces a deep state-space modeling (DSSM) framework tailored for monitoring complex and dynamic deteriorating systems operating under varying operating conditions and sensor-based condition monitoring...
详细信息
This article introduces a deep state-space modeling (DSSM) framework tailored for monitoring complex and dynamic deteriorating systems operating under varying operating conditions and sensor-based condition monitoring. By integrating stochastic recurrent neural networks (RNNs) with a generative model in a variational autoencoder (VAE) form, the proposed framework effectively approximates the complex behaviors inherent in degrading systems with latent states and captures long-term dependencies and latent dynamics without relying on unrealistic distributional and parametric assumptions. The framework leverages RNNs to model temporal dependencies and VAE to model robust probabilistic inference, enabling accurate latent state estimation and time to event (TE) prediction. Moreover, its flexibility extends to accommodating both continuous and discrete latent states, enriching the representation of underlying data dynamics. By performing joint inference and learning, utilizing VAE for system dynamics modeling offers significant advantages over traditional state-space models (SSMs), which require high computational resources and tuning. We have tested this framework using both simulation and real-world datasets. Also, a case study on a wind turbine dataset demonstrates the effectiveness of the proposed framework in early fault detection.
This study addresses the optimization of the Vehicle Routing Problem (VRP) with prioritized customers by introducing and comparing two advanced solution approaches: a metaheuristic-based hyperheuristic framework and a...
详细信息
This study addresses the optimization of the Vehicle Routing Problem (VRP) with prioritized customers by introducing and comparing two advanced solution approaches: a metaheuristic-based hyperheuristic framework and a variational autoencoder (VAE)-based hyperheuristic. The VRP with prioritized customers introduces additional complexity by requiring efficient routing while ensuring high-priority customers receive service within strict constraints. To tackle this challenge, the proposed metaheuristic-based hyperheuristic dynamically selects and adapts low-level heuristics using Simulated Annealing (SA) and Ant Colony Optimization (ACO), enhancing search efficiency and solution quality. In contrast, the VAE-based approach leverages deep learning to model historical routing patterns and autonomously generate new heuristics tailored to problem-specific characteristics. Through extensive computational experiments on benchmark VRP instances, our results reveal that both approaches significantly enhance routing efficiency, with the VAE-based method demonstrating superior generalization across varying problem structures. Specifically, the VAE-based approach reduces total travel costs by an average of 8% and improves customer priority satisfaction by 95% compared to traditional hyperheuristic methods. Moreover, a comparative analysis with recent state-of-the-art algorithms highlights the competitive performance of our approaches in balancing computational efficiency and solution quality. These findings underscore the potential of integrating metaheuristics with machine learning in complex routing problems and provide valuable insights for real-world logistics and transportation planning. Future research will explore the generalization of these methodologies to dynamic and large-scale routing scenarios.
Soil liquefaction assessment remains a crucial and complex challenge in seismic geotechnical engineering due to various liquefaction records and limited information, which entails a more generalized off-the-shelf mode...
详细信息
Soil liquefaction assessment remains a crucial and complex challenge in seismic geotechnical engineering due to various liquefaction records and limited information, which entails a more generalized off-the-shelf model that can achieve favourable performance on different data sources. In this work, a deep learning model is built and investigated on the soil liquefaction prediction and a modified transfer learning scheme between different data sources is presented. Various datasets, including shear wave velocity-based, CPT-based, SPT-based, and real cases, are collected and utilized to verify the effectiveness and accuracy of the proposed model. Because different data sources in soil liquefaction generally share several geotechnical and mechanical parameters, we work to combine model prior information, feature mapping and data reconstruction in transfer learning models to tackle multi-source domain adaption, which can be further applied to other predictive analysis and facilitate online learning models in geotechnical engineering. Also, the deep learning model is compared with several classical machine learning and ensemble learning models and the modified transfer learning model is formulated by comparing different feature transformation techniques integrated with various feature-based and instance-based transfer learning methods. The accuracy and effectiveness of the deep learning and modified transfer learning models have been validated in the numerical study.
Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in d...
详细信息
Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in discovering dependencies among labels. In this paper, motivated by the recent success of multi-view learning to improve the generalization performance, we propose a novel multi-view probabilistic model named latent conditional Bernoulli mixture (LCBM) for multi-label classification. LCBM is a generative model taking features from different views as inputs, and conditional on the latent subspace shared by the views a Bernoulli mixture model is adopted to build label dependencies. Inside each component of the mixture, the labels have a weak correlation which facilitates computational convenience. The mean field variational inference framework is used to carry out approximate posterior inference in the probabilistic model, where we propose a Gaussian mixture variational autoencoder (GMVAE) for effective posterior approximation. We further develop a scalable stochastic training algorithm for efficiently optimizing the model parameters and variational parameters, and derive an efficient prediction procedure based on greedy search. Experimental results on multiple benchmark datasets show that our approach outperforms other state-of-the-art methods under various metrics.
Accurate fault prediction of rolling bearing can predict the operation condition in advance, which is an important means to ensure the safety and reliability of rotating machinery. Aimed at the data processing of roll...
详细信息
Accurate fault prediction of rolling bearing can predict the operation condition in advance, which is an important means to ensure the safety and reliability of rotating machinery. Aimed at the data processing of rolling bearing vibration signal with multi-fault and long time series, an intelligent fault prediction model based on gate recurrent unit and hybrid autoencoder is proposed in this paper. Firstly, vibration signals of multi-faults are brought into multi-layer gate recurrent unit model for multi-step and multi-variable time series prediction. Secondly, variational autoencoder is used for data augmentation of fault samples. Thirdly, the augmented fault samples are brought into stacked denoising autoencoder for noise reduction and fault prediction. Finally, fault prediction results of rolling bearing can be achieved on the basis of gate recurrent unit and hybrid autoencoder of variational autoencoder and stacked denoising autoencoder. The bearing datasets of Case Western Reserve University are used to verify the effectiveness of the proposed method. Comparative experiment results show that the proposed fault prediction model has more accurate time series prediction result and higher fault prediction accuracy than other deep learning models. With 98.68% accuracy of fault prediction, the proposed fault prediction model can be taken as an effective tool for intelligent predictive maintenance of rolling bearing.
High labor costs and the requirement for significant domain expertise often result in a lack of anomaly labels in most time series. Consequently, employing unsupervised methods becomes critical for practical industria...
详细信息
High labor costs and the requirement for significant domain expertise often result in a lack of anomaly labels in most time series. Consequently, employing unsupervised methods becomes critical for practical industrial applications. However, prevailing reconstruction-based anomaly detection algorithms encounter challenges in capturing intricate underlying correlations and temporal dependencies in time series. This study introduces an unsupervised anomaly detection model called variational AutoeEncoder with Adversarial Training for Multivariate Time Series Anomaly Detection (VAEAT). Its fundamental concept involves adopting a two-phase training strategy to improve anomaly detection precision through adversarial reconstruction of raw data. In the first phase, the model reconstructs raw data to extract its basic features by training two enhanced variational autoencoders (VAEs) that incorporate both the long short -term memory (LSTM) network and the attention mechanism in their common encoder. In the second phase, the model refines reconstructed data to optimize the reconstruction quality. In this manner, this two-phase VAE model effectively captures intricate underlying correlation and temporal dependencies. A large number of experiments are conducted to evaluate the performance on five publicly available datasets, and experimental results illustrate that VAEAT exhibits robust performance and effective anomaly detection capabilities. The source code of the proposed VAEAT can be available at https://github .com /Du -Team /VAEAT.
Featured Application This study's findings hold significant implications for enhancing data privacy and utility in healthcare analytics. By evaluating synthetic data generation methods like CTGAN, TVAE, CopulaGAN ...
详细信息
Featured Application This study's findings hold significant implications for enhancing data privacy and utility in healthcare analytics. By evaluating synthetic data generation methods like CTGAN, TVAE, CopulaGAN and Copula across diverse medical datasets containing sensitive patient information, such as genetic profiles and medical histories, the research aims to improve the development of predictive models without compromising patient *** The generation of synthetic data holds significant promise for augmenting limited datasets while avoiding privacy issues, facilitating research, and enhancing machine learning models' robustness. Generative Adversarial Networks (GANs) stand out as promising tools, employing two neural networks-generator and discriminator-to produce synthetic data that mirrors real data distributions. This study evaluates GAN variants (CTGAN, CopulaGAN), a variational autoencoder, and copulas on diverse real datasets of different complexity encompassing numerical and categorical attributes. The results highlight CTGAN's sensitivity to training parameters and TVAE's robustness across datasets. Scalability challenges persist, with GANs demanding substantial computational resources. TVAE stands out for its high utility across all datasets, even for high-dimensional data, though it incurs higher privacy risks, which is indicative of the curse of dimensionality. While no single model universally excels, understanding the trade-offs and leveraging model strengths can significantly enhance synthetic data generation (SDG). Future research should focus on adaptive learning mechanisms, scalability enhancements, and standardized evaluation metrics to advance SDG methods effectively. Addressing these challenges will foster broader adoption and application of synthetic data.
暂无评论