Single image deraining (SIDR) often suffers from over/under deraining due to the nonuniformity of rain densities and the variety of raindrop scales. In this paper, we propose a continuous density-guided network (CODE-...
详细信息
Single image deraining (SIDR) often suffers from over/under deraining due to the nonuniformity of rain densities and the variety of raindrop scales. In this paper, we propose a continuous density-guided network (CODE-Net) for SIDR. Particularly, it is composed of a rain streak extractor and a denoiser, where the convolutional sparse coding (CSC) is exploited to filter out noises from the extracted rain streaks. Inspired by the reweighted iterative soft-threshold (ISTA) for CSC, we address the problem of continuous rain density estimation by learning the weights with channel attention blocks from sparse codes. We further develop a multiscale strategy to depict rain streaks appearing at different scales. Experiments on synthetic and real-world data demonstrate the superiority of our methods over recent state-of-the-arts, in terms of both quantitative and qualitative results. Additionally, instead of quantizing rain density with several levels, our CODE-Net can provide continuous-valued estimations of rain densities, which is more desirable in real applications.
Real-world situations often involve processing data from diverse imaging modalities like Multispectral (MS), Near Infrared (NIR), and RGB, each capturing different aspects of the same scene. These modalities often var...
详细信息
ISBN:
(纸本)9789464593617;9798331519773
Real-world situations often involve processing data from diverse imaging modalities like Multispectral (MS), Near Infrared (NIR), and RGB, each capturing different aspects of the same scene. These modalities often vary in spatial and spectral resolution. Hence, Multi-modal Image Super-Resolution (MISR) techniques are required to improve the spatial/spectral resolution of target modality, taking help from High Resolution (HR) guidance modality that shares common features like textures, edges, and other structures. Traditional MISR approaches using convolutional Neural Networks (CNNs) typically employ an encoder-decoder architecture, which is prone to overfit in data-limited scenarios. This work proposes a novel deep convolutional analysis sparsecoding method, utilizing convolutional transforms within a fusion framework that eliminates the need for a decoder network. Thus, reducing the trainable parameters and enhancing the suitability for data-limited application scenarios. A joint optimization framework is proposed, which learns deep convolutional transforms for Low Resolution (LR) images of the target modality and HR images of the guidance modality, along with a fusion transform that combines these transform features to reconstruct HR images of the target modality. In contrast to dictionary-based synthesis sparsecoding methods for MISR, the proposed approach offers improved performance with reduced complexity, leveraging the inherent advantages of transform learning. The efficacy of the proposed method is demonstrated using RGB-NIR and RGB-MS datasets, showing superior reconstruction performance compared to state-of-the-art techniques without introducing additional artifacts from the guidance image.
convolutional dictionary learning (CDL) is a widely used technique in computer vision for accurately capturing local features and texture information in signals. However, most existing CDL methods are based on batch p...
详细信息
ISBN:
(纸本)9798400712005
convolutional dictionary learning (CDL) is a widely used technique in computer vision for accurately capturing local features and texture information in signals. However, most existing CDL methods are based on batch processing, which requires computing the entire dataset at once, resulting in a significant memory requirement that limits further development. To address this issue, researchers have turned to online convolutional dictionary learning (OCDL) in recent years. OCDL inputs data as a stream and stores all historical information in a pair of fixed-size history arrays. This approach avoids the problem of increasing memory requirements with the number of samples. However, current OCDL methods encounter two primary issues when handling big datasets or large dictionaries: the high time complexity of updating history arrays or dictionaries and the challenge of selecting hyperparameters linked to the ADMM algorithm. This paper proposes a slice-based approximate OCDL method called SAOCDL to address the aforementioned problems. The algorithm utilizes a sparse approximation model, where historical data samples are approximated as the convolution sum of the best local dictionary of a single sample and the corresponding local sparse codes. The current sample and a pair of fixed-size history arrays that store this approximation are used to update the best local dictionary. The resulting optimization problem is solved in the spatial domain using the proposed convergent inertial proximal-gradient algorithm, which combines dry friction with Hessian-driven damping. Extensive experiments were conducted on various benchmark datasets, and the results demonstrate that the proposed SAOCDL approach is highly competitive in terms of performance, efficiency, and memory with state-of-the-art OCDL and CDL algorithms.
convolutional dictionary learning has become increasingly popular in signal and image processing for its ability to overcome the limitations of traditional patch-based dictionary learning. Although most studies on con...
详细信息
convolutional dictionary learning has become increasingly popular in signal and image processing for its ability to overcome the limitations of traditional patch-based dictionary learning. Although most studies on convolutional dictionary learning mainly focus on the unimodal case, real-world image processing tasks usually involve images from multiple modalities, e.g., visible and near-infrared (NIR) images. Thus, it is necessary to explore convolutional dictionary learning across different modalities. In this paper, we propose a novel multi-modal convolutional dictionary learning algorithm, which efficiently correlates different image modalities and fully considers neighborhood information at the image level. In this model, each modality is represented by two convolutional dictionaries, in which one dictionary is for common feature representation and the other is for unique feature representation. The model is constrained by the requirement that the convolutionalsparse representations (CSRs) for the common features should be the same across different modalities, considering that these images are captured from the same scene. We propose a new training method based on the alternating direction method of multipliers (ADMM) to alternatively learn the common and unique dictionaries in the discrete Fourier transform (DFT) domain. We show that our model converges in less than 20 iterations between the convolutional dictionary updating and the CSRs calculation. The effectiveness of the proposed dictionary learning algorithm is demonstrated on various multimodal image processing tasks, achieves better performance than both dictionary learning methods and deep learning based methods with limited training data.
In this paper, we propose to extend the standard convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be "low-rank" through a Canonical Polyadic decom...
详细信息
In this paper, we propose to extend the standard convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be "low-rank" through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation.
Measured intensity in high-energy monochromatic X-ray diffraction (HEXD) experiments provides information regarding the microstructure of the crystalline material under study. The location of intensity on an areal det...
详细信息
Measured intensity in high-energy monochromatic X-ray diffraction (HEXD) experiments provides information regarding the microstructure of the crystalline material under study. The location of intensity on an areal detector is determined by the lattice spacing and orientation of crystals so that changes in the heterogeneity of these quantities are reflected in the spreading of diffraction peaks over time. High temporal resolution of such dynamics can now be experimentally observed using technologies such as the mixed-mode pixel array detector (MM-PAD) which facilitates in situ dynamic HEXD experiments to study plasticity and its underlying mechanisms. In this paper, we define and demonstrate a feature computed directly from such diffraction time series data quantifying signal spread in a manner that is correlated with plastic deformation of the sample. A distinguishing characteristic of the analysis is the capability to describe the evolution from the distinct diffraction peaks of an undeformed alloy sample through to the non-uniform Debye-Scherrer rings developed upon significant plastic deformation. We build on our previous work modeling data using an overcomplete dictionary by treating temporal measurements jointly to improve signal spread recovery. We demonstrate our approach in simulations and on experimental HEXD measurements captured using the MM-PAD. Our method for characterizing the temporal evolution of signal spread is shown to provide an informative means of data analysis that adds to the capabilities of existing methods. Our work draws on ideas from convolutional sparse coding and requires solving a coupled convex optimization problem based on the alternating direction method of multipliers.
convolutional neural networks (CNNs) have been tremendously successful in solving imaging inverse problems. To understand their success, an effective strategy is to construct simpler and mathematically more tractable ...
详细信息
convolutional neural networks (CNNs) have been tremendously successful in solving imaging inverse problems. To understand their success, an effective strategy is to construct simpler and mathematically more tractable convolutional sparse coding (CSC) models that share essential ingredients with CNNs. Existing CSC methods, however, underperform leading CNNs in challenging inverse problems. We hypothesize that the performance gap may be attributed in part to how they process images at different spatial scales: While many CNNs use multiscale feature representations, existing CSC models mostly rely on single-scale dictionaries. To close the performance gap, we thus propose a multiscale convolutional dictionary structure. The proposed dictionary structure is derived from the U-Net, arguably the most versatile and widely used CNN for image-to-image learning problems. We show that incorporating the proposed multiscale dictionary in an otherwise standard CSC framework yields performance competitive with state-of-the-art CNNs across a range of challenging inverse problems including CT and MRI reconstruction. Our work thus demonstrates the effectiveness and scalability of the multiscale CSC approach in solving challenging inverse problems.
The problem of non-coherent direction of arrival (DOA) estimation of wideband signals with magnitude-only measurements is studied in this paper. Unlike the traditional coherent DOA estimation methods, where discrete F...
详细信息
ISBN:
(纸本)9781665451093
The problem of non-coherent direction of arrival (DOA) estimation of wideband signals with magnitude-only measurements is studied in this paper. Unlike the traditional coherent DOA estimation methods, where discrete Fourier Transform (DFT) can be applied to sensor measurements in order to formulate the problem into a narrowband form, the non-coherent model processes wideband signals in time domain directly by exploiting the convolutional sparse coding (CSC) framework. As shown by simulations, the proposed solution has the advantage of being robust against frequency independent sensor response errors.
Multispectral and hyperspectral image fusion (MS/HS fusion) aims to generate a high-resolution hyperspectral (HRHS) image by fusing a high-resolution multispectral (HRMS) and a low-resolution hyperspectral (LRHS) imag...
详细信息
Multispectral and hyperspectral image fusion (MS/HS fusion) aims to generate a high-resolution hyperspectral (HRHS) image by fusing a high-resolution multispectral (HRMS) and a low-resolution hyperspectral (LRHS) images. The deep unfolding-based MS/HS fusion method is a representative deep learning paradigm due to its excellent performance and sufficient interpretability. However, existing deep unfolding-based MS/HS fusion methods only rely on a fixed linear degradation model, which focuses on modeling the relationships between HRHS and HRMS, as well as HRHS and LRHS. In this paper, we break free from this observation model framework and propose a new observation model. Firstly, the proposed observation model is built based on the convolutional sparse coding (CSC) technique, and then a proximal gradient algorithm is designed to solve this model. Secondly, we unfold the iterative algorithm into a deep network, dubbed as MHF-CSCNet, where the proximal operators are learned using convolutional neural networks. Finally, all trainable parameters can be automatically learned end-to-end from the training pairs. Experimental evaluations conducted on various benchmark datasets demonstrate the superiority of our method both quantitatively and qualitatively compared to other state-of-the-art methods.
sparse representation (SR) is a widely accepted hyperspectral image (HSI) denoising model. Because of the curse of dimensionality and the desire to better fit the data, the SR models are typically deployed on small an...
详细信息
ISBN:
(数字)9781665427920
ISBN:
(纸本)9781665427920
sparse representation (SR) is a widely accepted hyperspectral image (HSI) denoising model. Because of the curse of dimensionality and the desire to better fit the data, the SR models are typically deployed on small and fully overlapping blocks whose results are averaged to produce the global denoised HSI. This "local-global" denoising mechanism ignores the dependencies between blocks, resulting in visual artifacts. This paper describes the underlying clean HSI with a 3D convolutional sparse coding (CSC) model, representing the HSI with a linear combination of few shift-invariant 3D spatial-spectral filters in a global dictionary. Instead of operating on patches, the CSC model sees the clean HSI is generated from a sum of local atoms that appear in a small number of locations throughout the image, naturally retaining the relationship between pixels. Moreover, we unfold the optimization process of the model into a spatial-spectral convolutionalsparse neural network which absorbs the interpretation ability of the model while supporting discriminative learning from data. Experimental results on both synthetic and real-world datasets show that our network achieves competitive denoising performances, qualitatively and quantitatively.
暂无评论