检索结果-内蒙古大学图书馆

PPT Fusion: Pyramid Patch Transformer for a Case Study in Image Fusion

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fu, Yu Xu, Tianyang Wu, Xiao-Jun Kittler, Josef Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science Jiangnan University Wuxi 214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

The Transformer architecture has witnessed a rapid development in recent years, outperforming the CNN architectures in many computer vision tasks, as exemplified by the Vision Transformers (ViT) for image classification. However, existing visual transformer models aim to extract semantic information for high-level tasks, such as classification and detection. These methods ignore the importance of the spatial resolution of the input image, thus sacrificing the local correlation information of neighboring pixels. In this paper, we propose a Patch Pyramid Transformer(PPT) to effectively address the above issues. Specifically, we first design a Patch Transformer to transform the image into a sequence of patches, where transformer encoding is performed for each patch to extract local representations. In addition, we construct a Pyramid Transformer to effectively extract the non-local information from the entire image. After obtaining a set of multi-scale, multi-dimensional, and multi-angle features of the original image, we design the image reconstruction network to ensure that the features can be reconstructed into the original input. To validate the effectiveness, we apply the proposed Patch Pyramid Transformer to image fusion tasks. The experimental results demonstrate its superior performance, compared to the state-of-the-art fusion approaches, achieving the best results on several evaluation indicators. Thanks to the underlying representational capacity of the PPT network, it can directly be applied to different image fusion tasks without redesigning or retraining the network. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep learning

RFN-Nest: An end-to-end residual fusion network for infrared and visible images

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Hui Wu, Xiao-Jun Kittler, Josef Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China The Center for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

In the image fusion field, the design of deep learning-based fusion methods is far from routine. It is invariably fusion-task specific and requires a careful consideration. The most difficult part of the design is to choose an appropriate strategy to generate the fused image for a specific task in hand. Thus, devising learnable fusion strategy is a very challenging problem in the community of image fusion. To address this problem, a novel end-to-end fusion network architecture (RFN-Nest) is developed for infrared and visible image fusion. We propose a residual fusion network (RFN) which is based on a residual architecture to replace the traditional fusion approach. A novel detail-preserving loss function, and a feature enhancing loss function are proposed to train RFN. The fusion model learning is accomplished by a novel two-stage training strategy. In the first stage, we train an auto-encoder based on an innovative nest connection (Nest) concept. Next, the RFN is trained using the proposed loss functions. The experimental results on public domain data sets show that, compared with the existing methods, our end-to-end fusion network delivers a better performance than the state-of-the-art methods in both subjective and objective evaluation. The code of our fusion method is available at https://***/hli1221/imagefusion-rfn-nest. © 2021, CC BY-NC-ND.

关键词： Image fusion

UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Rao, Dongyu Wu, Xiao-Jun Li, Hui Kittler, Josef Xu, Tianyang Jiangnan University Jiangsu Provincial Engineerinig Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science Lihu Avenue Wuxi214122 China University of Surrey Centre for Vision Speech and Signal Processing GuildfordGU2 7XH United Kingdom

In this paper, we propose a photorealistic style transfer network to emphasize the natural effect of photo realistic image stylization. In general, distortion of the image content and lacking of details are two typical issues in the style transfer field. To this end, we design a novel framework employing the U-Net structure to maintain the rich spatial clues, with a multi-layer feature aggregation (MFA) method to simultaneously provide the details obtained by the shallow layers in the stylization processing. In particular, an encoder based on the dense block and a decoder form a symmetrical structure of U-Net are jointly staked to realize an effective feature extraction and image reconstruc tion. Besides, a transfer module based on MFA and "adaptive instance normalization" (AdaIN) is inserted in the skip connection positions to achieve the stylization. Accordingly, the stylized image possesses the texture of a real photo and preserves rich content details without introducing any mask or post-processing steps. The experimental results on public datasets demonstrate that our method achieves a more faithful structural similarity with a lower style loss, reflecting the effectiveness and merit of our approach. © 2021, CC0.

关键词： Deep learning

Differentiable neural architecture learning for efficient neural network design

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Guo, Qingbei Wu, Xiao-Jun Kittler, Josef Feng, Zhiquan Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Shandong Provincial Key Laboratory of Network based Intelligent Computing University of Jinan Jinan250022 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Automated neural network design has received ever-increasing attention with the evolution of deep convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems that neural architecture search (NAS) confronts is that a large number of candidate neural architectures are required to train, using, for instance, reinforcement learning and evolutionary optimisation algorithms, at a vast computation cost. Even recent differentiable neural architecture search (DNAS) samples a small number of candidate neural architectures based on the probability distribution of learned architecture parameters to select the final neural architecture. To address this computational complexity issue, we introduce a novel architecture parameterisation based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to optimize the neural architecture without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of neural architecture by the continuous scaled sigmoid method with convergence guarantees. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of neural architecture search cost, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageN

关键词： Deep neural networks

Mind the gap: Enlarging the domain gap in open set domain adaptation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chang, Dongliang Sain, Aneeshan Ma, Zhanyu Song, Yi-Zhe Guo, Jun Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China Centre for Vision Speech and Signal Processing University of Surrey London United Kingdom

Unsupervised domain adaptation aims to leverage labeled data from a source domain to learn a classifier for an unlabeled target domain. Among its many variants, open set domain adaptation (OSDA) is perhaps the most challenging, as it further assumes the presence of unknown classes in the target domain. In this paper, we study OSDA with a particular focus on enriching its ability to traverse across larger domain gaps. Firstly, we show that existing state-of-the-art methods suffer a considerable performance drop in the presence of larger domain gaps, especially on a new dataset (PACS) that we re-purposed for OSDA. We then propose a novel framework to specifically address the larger domain gaps. The key insight lies with how we exploit the mutually beneficial information between two networks;(a) to separate samples of known and unknown classes, (b) to maximize the domain confusion between source and target domain without the influence of unknown samples. It follows that (a) and (b) will mutually supervise each other and alternate until convergence. Extensive experiments are conducted on Office-31, Office-Home, and PACS datasets, demonstrating the superiority of our method in comparison to other state-of-the-arts. Code available at https://***/dongliangchang/Mutual-to-Separate/ Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning

The devil is in the channels: Mutual-channel loss for fine-grained image classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chang, Dongliang Ding, Yifeng Xie, Jiyang Bhunia, Ayan Kumar Li, Xiaoxu Ma, Zhanyu Wu, Ming Guo, Jun Song, Yi-Zhe Pattern Recognition and Intelligent System Laboratory School of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing100876 China School of Computer and Communication Lanzhou University of Technology Lanzhou730050 China Centre for Vision Speech and Signal Processing University of Surrey London United Kingdom

The key to solving fine-grained image categorization is finding discriminate and local regions that correspond to subtle visual traits. Great strides have been made, with complex networks designed specifically to learn part-level discriminate feature representations. In this paper, we show that it is possible to cultivate subtle details without the need for overly complicated network designs or training mechanisms- a single loss is all it takes. The main trick lies with how we delve into individual feature channels early on, as opposed to the convention of starting from a consolidated feature map. The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component. The discriminality component forces all feature channels belonging to the same class to be discriminative, through a novel channel-wise attention mechanism. The diversity component additionally constraints channels so that they become mutually exclusive across the spatial dimension. The end result is therefore a set of feature channels, each of which reflects different locally discriminative regions for a specific class. The MC-Loss can be trained end-to-end, without the need for any boundingbox/ part annotations, and yields highly discriminative regions during inference. Experimental results show our MC-Loss when implemented on top of common base networks can achieve stateof- the-art performance on all four fine-grained categorization datasets (CUB-Birds, FGVC-Aircraft, Flowers-102, and Stanford Cars). Ablative studies further demonstrate the superiority of the MC-Loss when compared with other recently proposed generalpurpose losses for visual classification, on two different base networks. Code available at https://***/dongliangchang/ Mutual-Channel-Loss. Copyright © 2020, The Authors. All rights reserved.

关键词： Image classification

Self-grouping convolutional neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

Although group convolution operators are increasingly used in deep convolutional neural networks to improve the computational efficiency and to reduce the number of parameters, most existing methods construct their group convolution architectures by a predefined partitioning of the filters of each convolutional layer into multiple regular filter groups with an equal spatial group size and data-independence, which prevents a full exploitation of their potential. To tackle this issue, we propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN, in which the filters of each convolutional layer group themselves based on the similarity of their importance vectors. Concretely, for each filter, we first evaluate the importance value of their input channels to identify the importance vectors, and then group these vectors by clustering. Using the resulting data-dependent centroids, we prune the less important connections, which implicitly minimizes the accuracy loss of the pruning, thus yielding a set of diverse group convolution filters. Subsequently, we develop two fine-tuning schemes, i.e. (1) both local and global fine-tuning and (2) global only fine-tuning, which experimentally deliver comparable results, to recover the recognition capacity of the pruned network. Comprehensive experiments carried out on the CIFAR-10/100 and ImageNet datasets demonstrate that our self-grouping convolution method adapts to various state-of-the-art CNN architectures, (Figure presented) Figure 1: Evolution of group convolutions. (a) Regular convolution. (b) Regular group convolution. (c) Permuting group convolution. (d) Learned group convolution. (d) Self-grouping convolution. Note that white channels represent the ignored input channels, and gray channels indicate the reused input channels. such as ResNet and DenseNet, and delivers superior performance in terms of compression ratio, speedup and recognition accuracy. We demonstrate the ability of SG-CNN

关键词： Convolution

Affine Non-negative Collaborative Representation Based pattern Classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yin, He-Feng Wu, Xiao-Jun Feng, Zhen-Hua Kittler, Josef School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Department of Computer Science University of Surrey GuildfordGU2 7XH United Kingdom Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

—During the past decade, representation-based classification methods have received considerable attention in pattern recognition. In particular, the recently proposed non-negative representation based classification (NRC) method has been reported to achieve promising results in a wide range of classification tasks. However, NRC has two major drawbacks. First, there is no regularization term in the formulation of NRC, which may result in unstable solution and misclassification. Second, NRC ignores the fact that data usually lies in a union of multiple affine subspaces, rather than linear subspaces in practical applications. To address the above issues, this paper presents an affine non-negative collaborative representation (ANCR) model for pattern classification. To be more specific, ANCR imposes a regularization term on the coding vector. Moreover, ANCR introduces an affine constraint to better represent the data from affine subspaces. The experimental results on several benchmarking datasets demonstrate the merits of the proposed ANCR method. The source code of our ANCR is publicly available at https://***/yinhefeng/ANCR. Copyright © 2020, The Authors. All rights reserved.

关键词： Vectors

AIM 2020 Challenge on Image Extreme Inpainting 16th

学校读者我要写书评

暂无评论

AIM 2020 Challenge on Image Extreme Inpainting

Workshops held at the 16th European Conference on Computer Vision, ECCV 2020

作者： Ntavelis, Evangelos Romero, Andrés Bigdeli, Siavash Timofte, Radu Hui, Zheng Wang, Xiumei Gao, Xinbo Shin, Chajin Kim, Taeoh Son, Hanbin Lee, Sangyoun Li, Chao Li, Fu He, Dongliang Wen, Shilei Ding, Errui Bai, Mengmeng Li, Shuchen Zeng, Yu Lin, Zhe Yang, Jimei Zhang, Jianming Shechtman, Eli Lu, Huchuan Zeng, Weijian Ni, Haopeng Cai, Yiyang Li, Chenghua Xu, Dejia Wu, Haoning Han, Yu Nadim, Uddin S. M. Jang, Hae Woong Ahmed, Soikat Hasan Yoon, Jungmin Jung, Yong Ju Li, Chu-Tak Liu, Zhi-Song Wang, Li-Wen Siu, Wan-Chi Lun, Daniel P. K. Suin, Maitreya Purohit, Kuldeep Rajagopalan, A.N. Narang, Pratik Mandal, Murari Chauhan, Pranjal Singh Computer Vision Lab ETH Zürich Zürich Switzerland CSEM Neuchâtel Switzerland School of Electronic Engineering Xidian University Xi’an China Image and Video Pattern Recognition Laboratory School of Electrical and Electronic Engineering Yonsei University Seoul Korea Republic of Baidu Inc. Beijing China Beijing China Dalian University of Technology Dalian China Adobe San Jose United States Rensselaer Polytechnic Institute Troy United States Peking University Beijing China Lab Gachon University Seongnam Korea Republic of Centre for Multimedia Signal Processing Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hong Kong China Indian Institute of Technology Madras Chennai India BITS Pilani Pilani India MNIT Jaipur Jaipur India

ISBN: (纸本)9783030670696

This paper reviews the AIM 2020 challenge on extreme image inpainting. This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting. The goal of track 1 is to inpaint large part of the image with no supervision. Similarly, the goal of track 2 is to inpaint the image by having access to the entire semantic segmentation map of the input. The challenge had 88 and 74 participants, respectively. 11 and 6 teams competed in the final phase of the challenge, respectively. This report gauges current solutions and set a benchmark for future extreme image inpainting methods. © 2020, Springer Nature Switzerland AG.

关键词： Semantic Segmentation