We propose a generative model for music extension in this paper. The model is composed of two classifiers, one for music emotion and one for music tonality, and a generative adversarial network (GAN). Therefore, it ca...
详细信息
ISBN:
(纸本)9781728176055
We propose a generative model for music extension in this paper. The model is composed of two classifiers, one for music emotion and one for music tonality, and a generative adversarial network (GAN). Therefore, it can generate symbolic music not only based on low level spectral and temporal characteristics, but also on high level emotion and tonality attributes of previously observed music pieces. The generative model works in a universal latent space constructed by the variational autoencoder (VAE) for representing music pieces. We conduct subjective listening tests and derive objective measures for performance evaluation. Experimental results show that the proposed model produces much smoother and more authentic music pieces than the baseline model in terms of all subjective and objective measures.
Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets. They present, however, subtleties in training often manifesting in the discrete ...
详细信息
Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets. They present, however, subtleties in training often manifesting in the discrete latent variable not being leveraged. In this paper, we show why such models struggle to train using traditional log-likelihood maximization, and that they are amenable to training using the Optimal Transport framework of Wasserstein autoencoders. We find our discrete latent variable to be fully leveraged by the model when trained, without any modifications to the objective function or significant fine tuning. Our model generates comparable samples to other approaches while using relatively simple neural networks, since the discrete latent variable carries much of the descriptive burden. Furthermore, the discrete latent provides significant control over generation.
This paper describes a representation learning method for disentangling an arbitrary musical instrument sound into latent pitch and timbre representations. Although such pitch-timbre disentanglement has been achieved ...
详细信息
ISBN:
(纸本)9781728176055
This paper describes a representation learning method for disentangling an arbitrary musical instrument sound into latent pitch and timbre representations. Although such pitch-timbre disentanglement has been achieved with a variational autoencoder (VAE), especially for a predefined set of musical instruments, the latent pitch and timbre representations are outspread, making them hard to interpret. To mitigate this problem, we introduce a metric learning technique into a VAE with latent pitch and timbre spaces so that similar (different) pitches or timbres are mapped close to (far from) each other. Specifically, our VAE is trained with additional contrastive losses so that the latent distances between two arbitrary sounds of the same pitch or timbre are minimized, and those of different pitches or timbres are maximized. This training is performed under weak supervision that uses only whether the pitches and timbres of two sounds are the same or not, instead of their actual values. This improves the generalization capability for unseen musical instruments. Experimental results show that the proposed method can find better-structured disentangled representations with pitch and timbre clusters even for unseen musical instruments.
As opposed to group fairness algorithms which enforce equality of distributions, individual fairness aims at treating similar people similarly. In this paper, we focus on individual fairness regarding sensitive attrib...
详细信息
ISBN:
(纸本)9783030923068;9783030923075
As opposed to group fairness algorithms which enforce equality of distributions, individual fairness aims at treating similar people similarly. In this paper, we focus on individual fairness regarding sensitive attributes that should be removed from people comparisons. In that aim, we present a new method that leverages the variational autoencoder (VAE) algorithm and the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient for enforcing individual fairness in predictions. We also propose new metrics to assess individual fairness. We demonstrate the effectiveness of our approach in enforcing individual fairness on several machine learning tasks prone to algorithmic bias.
The treatment of interfering motion contributions remains one of the key challenges in the domain of radar-based vital sign monitoring. Removal of the interference to extract the vital sign contributions is demanding ...
详细信息
ISBN:
(纸本)9781728176093
The treatment of interfering motion contributions remains one of the key challenges in the domain of radar-based vital sign monitoring. Removal of the interference to extract the vital sign contributions is demanding due to overlapping Doppler bands, the complex structure of the interference motions and significant variations in the power levels of their contributions. A novel approach to the removal of interference through the use of a probabilistic deep learning model is presented. Results show that a convolutional encoder-decoder neural network with a variational objective is capable of learning a meaningful representation space of vital sign Doppler-time distribution facilitating their extraction from a mixture signal. The approach is tested on semi-experimental data containing real vital sign signatures and simulated returns from interfering body motions. It is demonstrated that the application of the proposed network enhances the extraction of the micro-Doppler frequency corresponding to the respiration rate.
In automotive digital development, engineers utilize multiple virtual prototyping tools to design and assess the performance of 3D shapes. However, accurate performance simulations are computationally expensive and ti...
详细信息
ISBN:
(纸本)9781728190488
In automotive digital development, engineers utilize multiple virtual prototyping tools to design and assess the performance of 3D shapes. However, accurate performance simulations are computationally expensive and time-consuming, which may be prohibitive for design optimization tasks. To address this challenge, we envision a 3D design assistance system for design exploration with performance assessment in the automotive domain. Recent advances in deep learning methods for learning geometric data are a promising step towards realizing such systems. Deep learning-based (variational) autoencoder models have been used for learning and compressing 3D data allowing engineers to generate low-dimensional representations of 3D designs. Finding representations in a data-driven fashion results in representations that are agnostic to downstream tasks performed on these representations and are believed to capture relevant design features. In this paper, we evaluate whether such data-driven representations contain relevant information about the input data and whether representations are meaningful in performance prediction tasks for the input data. We use machine learning-based surrogate models to predict the performances of car shapes based on the low-dimensional representation learned by 3D point cloud (variational) autoencoders. Furthermore, we exploit the stochastic nature of the representation learned by variational autoencoders to augment the training data for our surrogate models, since the limited amount of data is usually a challenge for surrogate modeling in engineering. We demonstrate that augmenting training with generated shapes improves prediction accuracy. In sum, we find that geometric deep learning approaches offer powerful tools to support the engineering design process.
variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of ge...
详细信息
ISBN:
(纸本)9781665438865
variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs for generating whole platformer and dungeon levels, and blending levels across these genres. We show that CVAEs can reliably control door placement in dungeons and progression direction in platformer levels. Thus, by using appropriate labels, our approach can generate whole dungeons and platformer levels of interconnected rooms and segments respectively as well as levels that blend dungeons and platformers. We demonstrate our approach using The Legend of Zelda, Metroid, Mega Man and Lode Runner.
In this paper, we propose a variational deep representation learning (VDRL) approach for cross-modal retrieval. Numerous existing methods map the image and text to the point representations, which is challenging to mo...
详细信息
ISBN:
(纸本)9783030880071;9783030880064
In this paper, we propose a variational deep representation learning (VDRL) approach for cross-modal retrieval. Numerous existing methods map the image and text to the point representations, which is challenging to model the semantic multiplicity of the sample. To address this issue, our VDRL aims to map the image and text to the semantic distributions and measure the similarity by comparing the difference between their distributions. Specifically, our VDRL network is trained under three constraints: 1) The variational autoencoder loss is minimized to learn the distributions of both the images in image semantic space and the texts in text semantic space. 2) The mutual information is introduced to ensure the VDRL learns the intact distribution for the sample. 3) The triplet hinge loss is incorporated to align the distributions of the images and texts at the semantic level. Consequently, the semantic multiplicity of each sample is modeled in our method. Experimental results demonstrate that our approach achieves compelling performance with state-of-the-art methods.
Generating explanations for recommendation systems is essential for improving its transparency since informative explanations such as generated reviews can help users comprehend the reason for receiving a specified re...
详细信息
ISBN:
(纸本)9781665423984
Generating explanations for recommendation systems is essential for improving its transparency since informative explanations such as generated reviews can help users comprehend the reason for receiving a specified recommendation. The generated reviews should be specific for the given user, item, and rating, however, recent works only focus on designing more and more powerful decoder, merely treating this task as a plain natural language generation process. We argue that there may exist the risk that the powerful decoder neglects the input embeddings and suffers from the biases that exist in data. In this paper, we propose a novel Injective variational autoencoders (InVAE) for generating high-quality reviews. Specifically, we employ a Collaborative Kullback-Leibler divergences (CKL) mechanism to building a better latent space that captures meaningful information. Base on this, the Spectral Regularization on Flow-based transformation (SRF) method is designed to backward transfer the priorities of generated latent variables to the input embeddings. Therefore, our method can construct more informative input embeddings and provides more specific explanations for different inputs. Extensive empirical experiments demonstrate that our model can construct much more meaningful feature embeddings and generate personalized reviews in high quality.
Tissue clearing and light-sheet microscopy technologies offer new opportunities to quantify the three-dimensional (3D) neural structure at a cellular or even sub-cellular resolution. Although many efforts have been ma...
详细信息
ISBN:
(纸本)9783030781903;9783030781910
Tissue clearing and light-sheet microscopy technologies offer new opportunities to quantify the three-dimensional (3D) neural structure at a cellular or even sub-cellular resolution. Although many efforts have been made to recognize nuclei in 3D using deep learning techniques, current state-of-the-art approaches often work in a two-step manner, i.e., first segment nucleus regions within a 2D optical slice and then assemble the regions into the 3D instance of a nucleus. Due to the poor inter-slice resolution in many volumetric microscopy images and lack of contextual information across image slices, the current two-step approaches yield less accurate instance segmentation results. To address these limitations, a novel neural network for 3D nucleus instance segmentation (NIS) is proposed, called NIS-Net, which jointly segments and assembles the 3D instances of nuclei. Specifically, a pretext task is designed to predict the image appearance of the to-be-processed slice using the learned context from the processed slices, where the well-characterized contextual information is leveraged to guide the assembly of 3D nuclei instances. Since our NIS-Net progressively identifies nuclei instances by sliding over the entire image stack, our method is capable of segmenting nuclei instances for the whole mouse brain. Experimental results show that our proposed NIS-Net achieves higher accuracy and more reasonable nuclei instances than the current counterpart methods.
暂无评论