vector quantized variational autoencoder (VQ-VAE) models realize fast image generation by encoding and quantifying the raw input in the single-level or hierarchical compressed latent space. However, the learned repres...
详细信息
ISBN:
(纸本)9781450386517
vector quantized variational autoencoder (VQ-VAE) models realize fast image generation by encoding and quantifying the raw input in the single-level or hierarchical compressed latent space. However, the learned representations are not expert in capturing complex relations existed, while one usually adopts domain-specific autoregressive models to fit a prior distribution for two stages of learning. In this work, we propose VQMG, a novel and unified framework for multi-hops relational reasoning and explicit representation learning. By introducing Multi-hops Graph Convolution Networks (MGCN), complicated relations from hierarchical latent space are effectively captured by Inner Graph, while the fitting of autoregressive prior are performed coherently by Outer Graph to promote the performance. Experiments on multimedia tasks including Point cloud segementation, Stroke-level text detection and Image generation verify the efficiency and applicability of our approach.
Background: In China, diabetes is a common, high-incidence chronic disease. Diabetes has become a severe public health problem. However, the current diagnosis and treatment methods are difficult to control the progres...
详细信息
Background: In China, diabetes is a common, high-incidence chronic disease. Diabetes has become a severe public health problem. However, the current diagnosis and treatment methods are difficult to control the progress of diabetes. Traditional Chinese Medicine (TCM) has become an option for the treatment of diabetes due to its low cost, good curative effect, and good ***: Based on the tongue images data to realize the fine classification of the diabetic population, provide a diagnostic basis for the formulation of individualized treatment plans for diabetes, ensure the accuracy and consistency of the TCM diagnosis, and promote the objective and standardized development of TCM diagnosis. Methods: We use the TFDA-1 tongue examination instrument to collect the tongue images of the subjects. Tongue Diagnosis Analysis System (TDAS) is used to extract the TDAS features of the tongue images. vector quantized variational autoencoder (VQ-VAE) extracts VQ-VAE features from tongue images. Based on VQ-VAE features, K -means clustering tongue images. TDAS features are used to describe the differences between clusters. Vision Transformer (ViT) combined with Grad-weighted Class Activation Mapping (Grad-CAM) is used to verify the clustering results and calculate positioning diagnostic ***: Based on VQ-VAE features, K-means divides the diabetic population into 4 clusters with clear boundaries. The silhouette, calinski harabasz, and davies bouldin scores are 0.391, 673.256, and 0.809, respectively. Cluster 1 had the highest Tongue Body L (TB-L) and Tongue Coating L (TC-L) and the lowest Tongue Coating Angular second moment (TC-ASM), with a pale red tongue and white coating. Cluster 2 had the highest TC-b with a yellow tongue coating. Cluster 3 had the highest TB-a with a red tongue. Group 4 had the lowest TB-L, TC-L, and TB-b and the highest Per-all with a purple tongue and the largest tongue coating area. ViT verifies the clustering results of K-m
In material design, the establishment of process-structure-property relationship is crucial for analyzing and controlling material microstructures. For the establishment of process-structure-property relationship, a c...
详细信息
In material design, the establishment of process-structure-property relationship is crucial for analyzing and controlling material microstructures. For the establishment of process-structure-property relationship, a central problem is the analysis, characterization, and control of microstructures, since microstructures are highly sensitive to material processing and critically affect material's properties. Therefore, accurately estimating the morphology of material microstructures plays a significant role in understanding the process-structure-property relationship. In this paper, we propose a deep-learning framework for estimating material microstructures under specific process conditions. The framework utilizes two deep learning networks: vector quantized variational autoencoder (VQVAE) and pixel convolutional neural network (PixelCNN). The framework can predict material microstructures from the transformation behavior given by some physical models. In this sense, the framework is consistent with the physical knowledge accumulated in the field of material science. Importantly, our study demonstrates qualitative and quantitative evidence that incorporating physical models enhances the accuracy of microstructure prediction by deep learning models. These results highlight the importance of appropriately integrating field-specific knowledge when applying data-driven frameworks to materials design. Consequently, our results provide a basis for integrating data-driven methods with the accumulated knowledge in the field. This integration holds great potential for advancing material design through deep learning.
Pathological brain appearances may be so heterogeneous as to be intelligible only as anomalies, defined by their deviation from normality rather than any specific set of pathological features. Amongst the hardest task...
详细信息
Pathological brain appearances may be so heterogeneous as to be intelligible only as anomalies, defined by their deviation from normality rather than any specific set of pathological features. Amongst the hardest tasks in medical imaging, detecting such anomalies requires models of the normal brain that combine compactness with the expressivity of the complex, long-range interactions that characterise its structural organisation. These are requirements transformers have arguably greater potential to satisfy than other current candidate architectures, but their application has been inhibited by their demands on data and computational resources. Here we combine the latent representation of vector quantised variationalautoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes. We compare our method to current state-of-the-art approaches across a series of experiments with 2D and 3D data involving synthetic and real pathological lesions. On real lesions, we train our models on 15,0 0 0 radiologically normal participants from UK Biobank and evaluate performance on four different brain MR datasets with small vessel disease, demyelinating lesions, and tumours. We demonstrate superior anomaly detection performance both image-wise and pixel/voxelwise, achievable without post-processing. These results draw attention to the potential of transformers in this most challenging of imaging tasks.
In material design, the establishment of process structure property relationship is crucial for analyzing and controlling material microstructures. For the establishment of process structure property relationship, a c...
详细信息
In material design, the establishment of process structure property relationship is crucial for analyzing and controlling material microstructures. For the establishment of process structure property relationship, a central problem is the analysis, characterization, and control of microstructures, since microstructures are highly sensitive to material processing and critically affect material's properties. Therefore, accurately estimating the morphology of material microstructures plays a significant role in understanding the process structure property relationship. In this paper, we propose a deep -learning framework for estimating material microstructures under specific process conditions. The framework utilizes two deep learning networks: vector quantized variational autoencoder (VQVAE) and Pixel Convolutional Neural Network (PixeICNN). The framework can predict material micrographs from the transformation behavior given by some physical model. In this sense, the framework is consistent with the physical knowledge accumulated in the field of material science. Importantly our study demonstrates qualitative and quantitative evidences that incorporating physical models enhances the accuracy of microstructure estimation by deep learning models. These results highlight the importance of appropriately integrating field -specific knowledge when applying data-driven frameworks to materials design. Consequently, our results provide a foundation for integrating data-driven methods with the accumulated knowledge in the field. This integration holds great potential for advancing material design through deep learning.
暂无评论