In many real-world inverse problems, only incomplete measurement data are available for training which can pose a problem for learning a reconstruction function. Indeed, unsupervised learning using a fixed incomplete ...
详细信息
ISBN:
(纸本)9781713871088
In many real-world inverse problems, only incomplete measurement data are available for training which can pose a problem for learning a reconstruction function. Indeed, unsupervised learning using a fixed incomplete measurement process is impossible in general, as there is no information in the nullspace of the measurement operator. This limitation can be overcome by using measurements from multiple operators. While this idea has been successfully applied in various applications, a precise characterization of the conditions for learning is still lacking. In this paper, we fill this gap by presenting necessary and sufficient conditions for learning the underlying signal model needed for reconstruction which indicate the interplay between the number of distinct measurement operators, the number of measurements per operator, the dimension of the model and the dimension of the signals. Furthermore, we propose a novel and conceptually simple unsupervised learning loss which only requires access to incomplete measurement data and achieves a performance on par with supervised learning when the sufficient condition is verified. We validate our theoretical bounds and demonstrate the advantages of the proposed unsupervised loss compared to previous methods via a series of experiments on various imaging inverse problems, such as accelerated magnetic resonance imaging, compressed sensing and image inpainting.
Traditional multi-view three-dimensional (3D) reconstruction uses images from visible light that may be of poor quality under conditions such as reflections, light spots, or haze, causing difficulties for 3D reconstru...
详细信息
The use of deep learning methods to decode visual perception images from brain activity recorded by fMRI has received a lot of attention. However, limited fMRI data make the task of visual reconstruction challenging. ...
详细信息
ISBN:
(纸本)9781665488679
The use of deep learning methods to decode visual perception images from brain activity recorded by fMRI has received a lot of attention. However, limited fMRI data make the task of visual reconstruction challenging. Inspired by hierarchical encoding of the visual cortex and the theory of brain homology with convolutional neural networks (CNNs), we propose a novel neural decoding model called hierarchical semantic generative adversarial network (HS-GAN). Specifically, we use CNN-based image encoder to extract hierarchical and semantic features of visually stimulus images. Then a neural decoder is used to decode hierarchical and semantic features from fMRI. In order to take full advantage of the information from different visual cortexes, we construct a generator with self-attention modules and skip connections to fuse the image features of different layers. In model training, adversarial learning is introduced to realize more natural imagereconstruction. Compared to existing advanced methods, our method significantly improves the naturalness and fidelity of reconstructed images.
Infrared and visible image fusion task aims to decompose and combine complementary information from both sensors. To overcome the lack of global intensity balance in fusion images, we proposed our joint transformer ne...
详细信息
ISBN:
(纸本)9798350385113;9798350385106
Infrared and visible image fusion task aims to decompose and combine complementary information from both sensors. To overcome the lack of global intensity balance in fusion images, we proposed our joint transformer network with feature enhancement and stack cross attention (SCA) layer. Firstly, axis-based self-attention layers are applied to extract shallow features. Then, feature enhancement layer extracts feature from spatial and channel perspectives into stacks. Subsequently, the SCA layer employs cross attention for feature interaction between modalities and cross-layer attention for reassembling feature stacks to targeted pattern, which adaptively generates cross modality and feature layer relationships, respectively. Moreover, to tackle the issue of maintaining fusion by results-oriented metrics, we conduct decomposition loss to constrain above procedure by controlling cross modality correlation. Therefore, modality-specific and modality-general features are divided properly, facilitating feature reconstruction in the decoder. Finally, qualitative results show that our method preserves abundant texture and precise intensity from source images. Quantitative experimental results demonstrate that our fusion network achieves the state-of-the-art fusion performance, especially in mutual information (MI).
Diffusion models have been demonstrated as powerful deep learning tools for image generation in CT reconstruction and restoration. Recently, diffusion posterior sampling, where a score-based diffusion prior is combine...
详细信息
ISBN:
(纸本)9781510671553;9781510671546
Diffusion models have been demonstrated as powerful deep learning tools for image generation in CT reconstruction and restoration. Recently, diffusion posterior sampling, where a score-based diffusion prior is combined with a likelihood model, has been used to produce high quality CT images given low-quality measurements. This technique is attractive since it permits a one-time, unsupervised training of a CT prior;which can then be incorporated with an arbitrary data model. However, current methods only rely on a linear model of x-ray CT physics to reconstruct or restore images. While it is common to linearize the transmission tomography reconstruction problem, this is an approximation to the true and inherently nonlinear forward model. We propose a new method that solves the inverse problem of nonlinear CT imagereconstruction via diffusion posterior sampling. We implement a traditional unconditional diffusion model by training a prior score function estimator, and apply Bayes rule to combine this prior with a measurement likelihood score function derived from the nonlinear physical model to arrive at a posterior score function that can be used to sample the reverse-time diffusion process. This plug-and-play method allows incorporation of a diffusion-based prior with generalized nonlinear CT imagereconstruction into multiple CT system designs with different forward models, without the need for any additional training. We demonstrate the technique in both fully sampled low dose data and sparse-view geometries using a single unsupervised training of the prior.
Finding an initial noise vector that produces an input image when fed into the diffusion process (known as inversion) is an important problem in denoising diffusion models (DDMs), with applications for real image edit...
详细信息
ISBN:
(纸本)9798350301298
Finding an initial noise vector that produces an input image when fed into the diffusion process (known as inversion) is an important problem in denoising diffusion models (DDMs), with applications for real image editing. The standard approach for real image editing with inversion uses denoising diffusion implicit models (DDIMs [29]) to deterministically noise the image to the intermediate state along the path that the denoising would follow given the original conditioning. However, DDIM inversion for real images is unstable as it relies on local linearization assumptions, which result in the propagation of errors, leading to incorrect imagereconstruction and loss of content. To alleviate these problems, we propose Exact Diffusion Inversion via Coupled Transformations (EDICT), an inversion method that draws inspiration from affine coupling layers. EDICT enables mathematically exact inversion of real and model-generated images by maintaining two coupled noise vectors which are used to invert each other in an alternating fashion. Using Stable Diffusion [25], a state-of-the-art latent diffusion model, we demonstrate that EDICT successfully reconstructs real images with high fidelity. On complex imagedatasets like MS-COCO, EDICT reconstruction significantly outperforms DDIM, improving the mean square error of reconstruction by a factor of two. Using noise vectors inverted from real images, EDICT enables a wide range of image edits-from local and global semantic edits to image stylization-while maintaining fidelity to the original image structure. EDICT requires no model training/finetuning, prompt tuning, or extra data and can be combined with any pretrained DDM.
While RAW images are efficient for image editing and perception tasks, their large size can strain camera storage and bandwidth. reconstruction methods of RAW images from sRGB data typically require additional metadat...
详细信息
High accuracy, low latency and high energy efficiency represent a set of conflicting goals when searching for system solutions for image classification and detection. While high-quality images naturally result in more...
详细信息
High Dynamic Range (HDR) imaging aims to replicate the high visual quality and clarity of real-world scenes. Due to the high costs associated with HDR imaging, the literature offers various data-driven methods for HDR...
详细信息
ISBN:
(纸本)9798350349405;9798350349399
High Dynamic Range (HDR) imaging aims to replicate the high visual quality and clarity of real-world scenes. Due to the high costs associated with HDR imaging, the literature offers various data-driven methods for HDR imagereconstructionfrom Low Dynamic Range (LDR) counterparts. A common limitation of these approaches is missing details in regions of the reconstructed HDR images, which are overor under-exposed in the input LDR images. To this end, we propose a simple and effective method, HistoHDR-Net, to recover the fine details (e.g., color, contrast, saturation, and brightness) of HDR images via a fusion-based approach utilizing histogram-equalized LDR images along with self-attention guidance. Our experiments demonstrate the efficacy of the proposed approach over the state-of-art methods.
This work aims to reconstruct accurate images from multiple images acquired by the Large Hadron Collider detector using advanced machine learning techniques. The importance of this task lies in enhancing the precision...
详细信息
暂无评论