image matching is a fundamental computer vision problem. While learning-based methods achieve state-of-the-art performance on existing benchmarks, they generalize poorly to in-the-wild images. Such methods typically n...
详细信息
image matching is a fundamental computer vision problem. While learning-based methods achieve state-of-the-art performance on existing benchmarks, they generalize poorly to in-the-wild images. Such methods typically need to train separate models for different scene types (e.g., indoor vs. outdoor) and are impractical when the scene type is unknown in advance. One of the underlying problems is the limited scalability of existing data construction pipelines, which limits the diversity of standard image matching datasets. To address this problem, we propose GIM, a self-training framework for learning a single generalizable model based on any image matching architecture using internet videos, an abundant and diverse data source. Given an architecture, GIM first trains it on standard domain-specific datasets and then combines it with complementary matching methods to create dense labels on nearby frames of novel videos. These labels are filtered by robust fitting, and then enhanced by propagating them to distant frames. The final model is trained on propagated data with strong augmentations. Not relying on complex 3D reconstruction makes GIM much more efficient and less likely to fail than standard SfM-and-MVS based frameworks. We also propose ZEB, the first zero-shot evaluation benchmark for image matching. By mixing datafrom diverse domains, ZEB can thoroughly assess the cross-domain generalization performance of different methods. Experiments demonstrate the effectiveness and generality of GIM. Applying GIM consistently improves the zero-shot performance of 3 state-of-the-art image matching architectures as the number of downloaded videos increases (Fig. 1 (a));with 50 hours of YouTube videos, the relative zero-shot performance improves by 6.9% - 18.1%. GIM also enables generalization to extreme cross-domain data such as Bird Eye View (BEV) images of projected 3D point clouds (Fig. 1 (c)). More importantly, our single zero-shot model consistently outperforms domain-s
image restoration has several uses, remote sensing, surveillance, medical imaging, and computational photography. Recovering high-quality image content from lower-quality images is its goal. Convolutional neural netwo...
详细信息
Accurate 3D cardiac reconstructionfrom cine magnetic resonance imaging (cMRI) is crucial for improved cardiovascular disease diagnosis and understanding of the heart's motion. However, current cardiac MRI-based r...
详细信息
ISBN:
(纸本)9783031439896;9783031439902
Accurate 3D cardiac reconstructionfrom cine magnetic resonance imaging (cMRI) is crucial for improved cardiovascular disease diagnosis and understanding of the heart's motion. However, current cardiac MRI-based reconstruction technology used in clinical settings is 2D with limited through-plane resolution, resulting in low-quality reconstructed cardiac volumes. To better reconstruct 3D cardiac volumes from sparse 2D image stacks, we propose a morphology-guided diffusion model for 3D cardiac volume reconstruction, DMCVR, that synthesizes high-resolution 2D images and corresponding 3D reconstructed volumes. Our method outperforms previous approaches by conditioning the cardiac morphology on the generative model, eliminating the time-consuming iterative optimization process of the latent code, and improving generation quality. The learned latent spaces provide global semantics, local cardiac morphology and details of each 2D cMRI slice with highly interpretable value to reconstruct 3D cardiac shape. Our experiments show that DMCVR is highly effective in several aspects, such as 2D generation and 3D reconstruction performance. With DMCVR, we can produce high-resolution 3D cardiac MRI reconstructions, surpassing current techniques. Our proposed framework has great potential for improving the accuracy of cardiac disease diagnosis and treatment planning. Code can be accessed at https://***/hexiaoxiao- cs/DMCVR.
The proceedings contain 19 papers. The special focus in this conference is on Simulation and Synthesis in Medical Imaging. The topics include: Adapted nnU-Net: A Robust Baseline for Cross-Modality Synthesis and...
ISBN:
(纸本)9783031732805
The proceedings contain 19 papers. The special focus in this conference is on Simulation and Synthesis in Medical Imaging. The topics include: Adapted nnU-Net: A Robust Baseline for Cross-Modality Synthesis and Medical image Inpainting;Beyond MR image Harmonization: Resolution Matters Too;benchmarking Robustness of Endoscopic Depth Estimation with Synthetically Corrupted data;a Dual-Task Mutual Learning Framework for Predicting Post-thrombectomy Cerebral Hemorrhage;TSynD: Targeted Synthetic data Generation for Enhanced Medical image Classification: Leveraging Epistemic Uncertainty to Improve Model Performance;beyond Intensity Transforms: Medical image Synthesis Under Large Deformation;sim2Real in Endoscopy Segmentation with a Novel Structure Aware image Translation;fireflies: Photorealistic Simulation and Optimization of Structured Light Endoscopy;exogenous Agent-Free Synthetic Post-contrast Imaging with a Cascade of Deep Networks for Enhancement Prediction After Tumor Resection. A Parametric-Map Oriented Approach;OCT Scans Simulation Framework for data Augmentation and Controlled Evaluation of Signal Processing Approaches;enhancing Quantitative image Synthesis Through Pretraining and Resolution Scaling for Bone Mineral Density Estimation from a Plain X-Ray image;latent Pollution Model: The Hidden Carbon Footprint in 3D image Synthesis;Synthesizing Scalable CFD-Enhanced Aortic 4D Flow MRI for Assessing Accuracy and Precision of Deep-Learning imagereconstruction and Segmentation Tasks;MedEdit: Counterfactual Diffusion-Based image Editing on Brain MRI;Using MR Physics for Domain Generalisation and Super-Resolution;Single-Scan mpMRI Calibration of Multi-species Brain Tumor Dynamics with Mass Effect;annotated Biomedical Video Generation Using Denoising Diffusion Probabilistic Models and Flow Fields.
In this paper, we develop a new method to automatically convert 2D line drawings from three orthographic views into 3D CAD models. Existing methods for this problem reconstruct 3D models by back-projecting the 2D obse...
ISBN:
(纸本)9798350307184
In this paper, we develop a new method to automatically convert 2D line drawings from three orthographic views into 3D CAD models. Existing methods for this problem reconstruct 3D models by back-projecting the 2D observations into 3D space while maintaining explicit correspondence between the input and output. Such methods are sensitive to errors and noises in the input, thus often fail in practice where the input drawings created by human designers are imperfect. To overcome this difficulty, we leverage the attention mechanism in a Transformer-based sequence generation model to learn flexible mappings between the input and output. Further, we design shape programs which are suitable for generating the objects of interest to boost the reconstruction accuracy and facilitate CAD modeling applications. Experiments on a new benchmark dataset show that our method significantly outperforms existing ones when the inputs are noisy or incomplete.
image synthesis in the context of radio interferometric data can be expressed as a signal reconstructionfromincomplete Fourier measurements. Most imaging techniques for radio interferometry lie in minimizing the lea...
详细信息
ISBN:
(数字)9781665485241
ISBN:
(纸本)9781665485241
image synthesis in the context of radio interferometric data can be expressed as a signal reconstructionfromincomplete Fourier measurements. Most imaging techniques for radio interferometry lie in minimizing the least square error between the reconstructed image and the observed data assuming an additive white gaussian noise. In this paper, we derive an expectation-maximization based imaging algorithm that handles the presence of outliers in the observed data. Subsequently, we propose a new generic image synthesis algorithm based on the expectation-maximization algorithm, leading to a computationally efficient method.
In this study, we propose a new tomographic reconstruction method for high-resolution samples based on the MapReduce model. We executed the method in a big data environment with a cluster installed on the Amazon Web S...
详细信息
ISBN:
(纸本)9781665434188
In this study, we propose a new tomographic reconstruction method for high-resolution samples based on the MapReduce model. We executed the method in a big data environment with a cluster installed on the Amazon Web Services (AWS) platform. The big data environment framework considered four sets of matrices from a single heterogeneous plexiglass phantom sample, totaling 7,840 matrices (35.63 GB) processed by 12 different frameworks and producing 427.56 GB of processed tomographic data. The proposed method enabled the analysis of large numbers of agricultural samples using X-ray tomography to support management based on precision agriculture paradigms, the decision-making processes of which require an increasing number of analyses. Index Terms-tomographic
The proceedings contain 21 papers. The special focus in this conference is on Deep Generative Models for Medical image Computing and Computer Assisted Intervention. The topics include: Energy-Based Prior Latent Space ...
ISBN:
(纸本)9783031727436
The proceedings contain 21 papers. The special focus in this conference is on Deep Generative Models for Medical image Computing and Computer Assisted Intervention. The topics include: Energy-Based Prior Latent Space Diffusion Model for reconstruction of Lumbar Vertebrae from Thick Slice MRI;anatomically-Guided Inpainting for Local Synthesis of Normal Chest Radiographs;enhancing Cross-Modal Medical image Segmentation Through Compositionality;unpaired Modality Translation for Pseudo Labeling of Histology images;SNAFusion: Distilling 2D Axial Plane Diffusion Priors for Sparse-View 3D Cone-Beam CT Imaging;SynthBrainGrow: Synthetic Diffusion Brain Aging for Longitudinal MRI data Generation in Young People;denoising Diffusion Models for 3D Healthy Brain Tissue Inpainting;panoptic Segmentation of Mammograms with Text-to-image Diffusion Model;interactive Generation of Laparoscopic Videos with Diffusion Models;Multi-parametric MRI to FMISO PET Synthesis for Hypoxia Prediction in Brain Tumors;qMRI Diffuser: Quantitative T1 Mapping of the Brain Using a Denoising Diffusion Probabilistic Model;on Differentially Private 3D Medical image Synthesis with Controllable Latent Diffusion Models;five Pitfalls When Assessing Synthetic Medical images with Reference Metrics;Augmenting Prostate MRI dataset with Synthetic Volumetric images from Zone-Conditioned Diffusion Generative Model;tiBiX: Leveraging Temporal Information for Bidirectional X-Ray and Report Generation;Segmentation-Guided MRI reconstruction for Meaningfully Diverse reconstructions;Non-reference Quality Assessment for Medical Imaging: Application to Synthetic Brain MRIs;latentArtiFusion: An Effective and Efficient Histological Artifacts Restoration Framework;How to Segment in 3D Using 2D Models: Automated 3D Segmentation of Prostate Cancer Metastatic Lesions on PET Volumes Using Multi-angle Maximum Intensity Projections and Diffusion Models.
Although augmentations (e.g., perturbation of graph edges, image crops) boost the efficiency of Contrastive Learning (CL), feature level augmentation is another plausible, complementary yet not well researched strateg...
详细信息
ISBN:
(纸本)9781577358800
Although augmentations (e.g., perturbation of graph edges, image crops) boost the efficiency of Contrastive Learning (CL), feature level augmentation is another plausible, complementary yet not well researched strategy. Thus, we present a novel spectral feature argumentation for contrastive learning on graphs (and images). To this end, for each data view, we estimate a low-rank approximation per feature map and subtract that approximation from the map to obtain its complement. This is achieved by the proposed herein incomplete power iteration, a non-standard power iteration regime which enjoys two valuable byproducts (under mere one or two iterations): (i) it partially balances spectrum of the feature map, and (ii) it injects the noise into rebalanced singular values of the feature map (spectral augmentation). For two views, we align these rebalanced feature maps as such an improved alignment step can focus more on less dominant singular values of matrices of both views, whereas the spectral augmentation does not affect the spectral angle alignment (singular vectors are not perturbed). We derive the analytical form for: (i) the incomplete power iteration to capture its spectrum-balancing effect, and (ii) the variance of singular values augmented implicitly by the noise. We also show that the spectral augmentation improves the generalization bound. Experiments on graph/imagedatasets show that our spectral feature augmentation outperforms baselines, and is complementary with other augmentation strategies and compatible with various contrastive losses.
In this work, we present a solution to the challenging problem of reconstructing liquids fromimagedata. The challenges in reconstructing liquids, which is not faced in previous reconstruction works on rigid and defo...
详细信息
ISBN:
(数字)9781665469463
ISBN:
(纸本)9781665469463
In this work, we present a solution to the challenging problem of reconstructing liquids fromimagedata. The challenges in reconstructing liquids, which is not faced in previous reconstruction works on rigid and deforming surfaces, lies in the inability to use depth sensing and color features due the variable index of refraction, opacity, and environmental reflections. Therefore, we limit ourselves to only surface detections (i.e. binary mask) of liquids as observations and do not assume any prior knowledge on the liquids properties. A novel optimization problem is posed which reconstructs the liquid as particles by minimizing the error between a rendered surface from the particles and the surface detections while satisfying liquid constraints. Our solvers to this optimization problem are presented and no training data is required to apply them. We also propose a dynamic prediction to seed the reconstruction optimization from the previous time-step. We test our proposed methods in simulation and on two new liquid datasets which we open source1 so the broader research community can continue developing in this under explored area.
暂无评论