检索结果-内蒙古大学图书馆

RFN-Nest: An end-to-end residual fusion network for infrared and visible images

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Hui Wu, Xiao-Jun Kittler, Josef Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China The Center for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

In the image fusion field, the design of deep learning-based fusion methods is far from routine. It is invariably fusion-task specific and requires a careful consideration. The most difficult part of the design is to choose an appropriate strategy to generate the fused image for a specific task in hand. Thus, devising learnable fusion strategy is a very challenging problem in the community of image fusion. To address this problem, a novel end-to-end fusion network architecture (RFN-Nest) is developed for infrared and visible image fusion. We propose a residual fusion network (RFN) which is based on a residual architecture to replace the traditional fusion approach. A novel detail-preserving loss function, and a feature enhancing loss function are proposed to train RFN. The fusion model learning is accomplished by a novel two-stage training strategy. In the first stage, we train an auto-encoder based on an innovative nest connection (Nest) concept. Next, the RFN is trained using the proposed loss functions. The experimental results on public domain data sets show that, compared with the existing methods, our end-to-end fusion network delivers a better performance than the state-of-the-art methods in both subjective and objective evaluation. The code of our fusion method is available at https://***/hli1221/imagefusion-rfn-nest. © 2021, CC BY-NC-ND.

关键词： Image fusion

Self-grouping convolutional neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Guo, Qingbei Wu, Xiao-Jun Kittler, Josef Feng, Zhiquan Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Shandong Provincial Key Laboratory of Network based Intelligent Computing University of Jinan Jinan250022 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Although group convolution operators are increasingly used in deep convolutional neural networks to improve the computational efficiency and to reduce the number of parameters, most existing methods construct their group convolution architectures by a predefined partitioning of the filters of each convolutional layer into multiple regular filter groups with an equal spatial group size and data-independence, which prevents a full exploitation of their potential. To tackle this issue, we propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN, in which the filters of each convolutional layer group themselves based on the similarity of their importance vectors. Concretely, for each filter, we first evaluate the importance value of their input channels to identify the importance vectors, and then group these vectors by clustering. Using the resulting data-dependent centroids, we prune the less important connections, which implicitly minimizes the accuracy loss of the pruning, thus yielding a set of diverse group convolution filters. Subsequently, we develop two fine-tuning schemes, i.e. (1) both local and global fine-tuning and (2) global only fine-tuning, which experimentally deliver comparable results, to recover the recognition capacity of the pruned network. Comprehensive experiments carried out on the CIFAR-10/100 and ImageNet datasets demonstrate that our self-grouping convolution method adapts to various state-of-the-art CNN architectures, (Figure presented) Figure 1: Evolution of group convolutions. (a) Regular convolution. (b) Regular group convolution. (c) Permuting group convolution. (d) Learned group convolution. (d) Self-grouping convolution. Note that white channels represent the ignored input channels, and gray channels indicate the reused input channels. such as ResNet and DenseNet, and delivers superior performance in terms of compression ratio, speedup and recognition accuracy. We demonstrate the ability of SG-CNN

关键词： Convolution

Learning a representation with the block-diagonal structure for pattern classification

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Yin, He-Feng Wu, Xiao-Jun Kittler, Josef Feng, Zhen-Hua School of Internet of Things Engineering Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Sparse-representation-based classification (SRC) has been widely studied and developed for various practical signal classification applications. However, the performance of a SRC-based method is degraded when both the training and test data are corrupted. To counteract this problem, we propose an approach that learns Representation with Block-Diagonal Structure (RBDS) for robust image recognition. To be more specific, we first introduce a regularization term that captures the block-diagonal structure of the target representation matrix of the training data. The resulting problem is then solved by an optimizer. Last, based on the learned representation, a simple yet effective linear classifier is used for the classification task. The experimental results obtained on several benchmarking datasets demonstrate the efficacy of the proposed RBDS method. The code will be released upon the acceptance of this paper. Copyright © 2019, The Authors. All rights reserved.

关键词： Image recognition

Multiple riemannian manifold-valued descriptors based image set classification with multi-kernel metric learning

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wang, Rui Wu, Xiao-Jun Kittler, Josef School of Internet of Things Engineering Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

The importance of wild video based image set recognition is becoming monotonically increasing due to the large amount of video resources obtained by diversified video collection approaches, like surveillance, drive recorders, smart phones, and internet. However, the contents of these collected videos are often complicated, and how to efficiently perform set modeling and feature extraction is a big challenge for set-based classification algorithms. In recent years, some proposed image set classification methods have made a considerable advance by modeling the original image set with covariance matrix, linear subspace, or Gaussian distribution. Moreover, the distinctive geometry spanned by them are Symmetric Positive Definite (SPD) manifold, Grassmann manifold, and Gaussian embedded Riemannian manifold, respectively. As a matter of fact, most of them just adopt a single geometric model to describe each given image set, which may lose some other useful information for classification. To tackle this problem, we propose a novel algorithm to model each image set from a multi-geometric perspective. Specifically, the covariance matrix, linear subspace, and Gaussian distribution are applied for set representation simultaneously. In order to fuse these multiple heterogeneous Riemannian manifoldvalued features, the well-equipped Riemannian kernel functions are first utilized to map them into high dimensional Hilbert spaces. Then, a multi-kernel metric learning framework is devised to embed the learned hybrid kernels into a lower dimensional common subspace for classification. We conduct experiments on four widely used datasets corresponding to four different classification tasks: video-based face recognition, set-based object categorization, video-based emotion recognition, and dynamic scene classification, to evaluate the classification performance of the proposed algorithm. Extensive experimental results justify its superiority over the state-of-the-art. Copyright © 2019, Th

关键词： Vector spaces

Multi-scale Frequency Enhancement Network for Blind Image Deblurring

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xiang, Yawen Zhou, Heng Li, Chengyang Li, Zhongbo Xie, Yongqiang The Institute of Systems Engineering Academy of Military Science Beijing100141 China School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China Beijing100100 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Beijing100100 China

Image deblurring is an essential image preprocessing technique, aiming to recover clear and detailed images form blurry ones. However, existing algorithms often fail to effectively integrate multi-scale feature extraction with frequency enhancement, limiting their ability to reconstruct fine textures. Additionally, non-uniform blur in images also restricts the effectiveness of image restoration. To address these issues, we propose a multi-scale frequency enhancement network (MFENet) for blind image deblurring. To capture the multi-scale spatial and channel information of blurred images, we introduce a multi-scale feature extraction module (MS-FE) based on depthwise separable convolutions, which provides rich target features for deblurring. We propose a frequency enhanced blur perception module (FEBP) that employs wavelet transforms to extract high-frequency details and utilizes multi-strip pooling to perceive non-uniform blur, combining multiscale information with frequency enhancement to improve the restoration of image texture details. Experimental results on the GoPro and HIDE datasets demonstrate that the proposed method achieves superior deblurring performance in both visual quality and objective evaluation metrics. Furthermore, in downstream object detection tasks, the proposed blind image deblurring algorithm significantly improves detection accuracy, further validating its effectiveness androbustness in the field of image deblurring. Copyright © 2024, The Authors. All rights reserved.

关键词： Convolution

Convolutional Neural Network-Based Multimodal Image Information Fusion for Moisture Damage Assessment of Cultural Heritage Buildings

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Wang, Fuzhi Huang, Jizhong Fu, Yu Department of Civil Engineering Shanghai University Shanghai200444 China Institute for the Conservation of Cultural Heritage Shanghai University Shanghai200444 China Key Laboratory of Silicate Cultural Relics Conservation Shanghai University Ministry of Education Shanghai200444 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China

Water is an essential factor causing the deterioration of historical and cultural heritage, and identifying and quantifying moisture damage is essential for conserving cultural heritage buildings. This study combines infrared thermography (IRT) and microwave technology to investigate moisture-induced diseases and the variation of the spatial distribution of water content in the Yungang Grottoes of China, a World Heritage Site. Information on building humidity distribution was extracted and combined from IRT and microwave information. Furthermore, multi-feature image fusion combines different types of monitoring information through the Convolutional Neural Network (CNN), which is used to diagnose moisture distribution and content accurately. The features obtained from different non-destructive methods were then jointly analyzed using statistical analysis methods such as absolute contrast, noise ratio (SNR), and spatial stability to quantify the artifact moisture defect data, thus improving the accuracy and range of features for detecting water-damaged pathologies. © 2024, The Authors. All rights reserved.

关键词： Convolution

Riemannian Self-Attention Mechanism for SPD Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Rui Wu, Xiao-Jun Li, Hui Kittler, Josef School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Symmetric positive definite (SPD) matrix has been demonstrated to be an effective feature descriptor in many scientific areas, as it can encode spatiotemporal statistics of the data adequately on a curved Riemannian manifold, i.e., SPD manifold. Although there are many different ways to design network architectures for SPD matrix nonlinear learning, very few solutions explicitly mine the geometrical dependencies of features at different layers. Motivated by the great success of self-attention mechanism in capturing long-range relationships, an SPD manifold self-attention mechanism (SMSA) is proposed in this paper using some manifold-valued geometric operations, mainly the Riemannian metric, Riemannian mean, and Riemannian optimization. Then, an SMSA-based geometric learning module (SMSA-GLM) is designed for the sake of improving the discrimination of the generated deep structured representations. Extensive experimental results achieved on three benchmarking datasets show that our modification against the baseline network further alleviates the information degradation problem and leads to improved accuracy. Copyright © 2023, The Authors. All rights reserved.

关键词： Network architecture

Adaptive multi-modal fusion hashing via hadamard matrix

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yu, Jun Zhang, Donglin Shu, Zhenqiu Chen, Feng The College of Computer and Communication Engineering Zhengzhou University of Light Industry Zhengzhou China The Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China The Faculty of Information Engineering and Automation Kunming University of Science and Technology Kunming China The School of Computer Science and Technology Anhui University of Technology Ma'anshan China

Hashing plays an important role in information retrieval, due to its low storage and high speed of processing. Among the techniques available in the literature, multi-modal hashing, which can encode heterogeneous multi-modal features into compact hash codes, has received particular attention. Most of the existing multi-modal hashing methods adopt the fixed weighting factors to fuse multiple modalities for any query data, which cannot capture the variation of different queries. Besides, many methods introduce hyper-parameters to balance many regularization terms that make the optimization harder. Meanwhile, it is time-consuming and labor-intensive to set proper parameter values. The limitations may significantly hinder their promotion in real applications. In this paper, we propose a simple, yet effective method that is inspired by the Hadamard matrix. The proposed method captures the multi-modal feature information in an adaptive manner and preserves the discriminative semantic information in the hash codes. Our framework is flexible and involves a very few hyper-parameters. Extensive experimental results show the method is effective and achieves superior performance compared to state-of-the-art algorithms. Copyright © 2020, The Authors. All rights reserved.

关键词： Hash functions

A localization method avoiding flip ambiguities for micro-UAVs with bounded distance measurement errors

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Guo, Qingbei Zhang, Yuan Lloret, Jaime Kantarci, Burak Seah, Winston K.G. Shandong Provincial Key Laboratory of Network Based Intelligent Computing University of Jinan Jinan Shandong China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Integrated Management Coastal Research Institute Universidad Politecnica de Valencia Spain School of Electrical Engineering and Computer Science University of Ottawa Canada School of Engineering and Computer Science Victoria University of Wellington New Zealand

Localization is a fundamental function in cooperative control of micro unmanned aerial vehicles (UAVs), but is easily affected by flip ambiguities because of measurement errors and flying motions. This study proposes a localization method that can avoid the occurrence of flip ambiguities in bounded distance measurement errors and constrained flying motions;to demonstrate its efficacy, the method is implemented on bilateration and trilateration. For bilateration, an improved bi-boundary model based on the unit disk graph model is created to compensate for the shortage of distance constraints, and two boundaries are estimated as the communication range constraint. The characteristic of the intersections of the communication range and distance constraints is studied to present a unique localization criterion which can avoid the occurrence of flip ambiguities. Similarly, for trilateration, another unique localization criterion for avoiding flip ambiguities is proposed according to the characteristic of the intersections of three distance constraints. The theoretical proof shows that these proposed criteria are correct. A localization algorithm is constructed based on these two criteria. The algorithm is validated using simulations for different scenarios and parameters, and the proposed method is shown to provide excellent localization performance in terms of average estimated error. Our code can be found at: https://***/QingbeiGuo/***. Copyright © 2018, The Authors. All rights reserved.

关键词： Measurement errors