检索结果-内蒙古大学图书馆

Dual encoder-decoder based generative adversarial networks for disentangled facial representation learning

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Hu, Cong Feng, Zhen-Hua Wu, Xiao-Jun Kittler, Josef School of Internet of Things Engineering Jiangnan University Wuxi Jiangsu Province214122 China Jiangsu Provincial Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi Jiangsu Province214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

To learn disentangled representations of facial images, we present a Dual Encoder-Decoder based Generative Adversarial Network (DED-GAN). In the proposed method, both the generator and discriminator are designed with deep encoder-decoder architectures as their backbones. To be more specific, the encoder-decoder structured generator is used to learn a pose disentangled face representation, and the encoder-decoder structured discriminator is tasked to perform real/fake classification, face reconstruction, determining identity and estimating face pose. We further improve the proposed network architecture by minimising the additional pixel-wise loss defined by the Wasserstein distance at the output of the discriminator so that the adversarial framework can be better trained. Additionally, we consider face pose variation to be continuous, rather than discrete in existing literature, to inject richer pose information into our model. The pose estimation task is formulated as a regression problem, which helps to disentangle identity information from pose variations. The proposed network is evaluated on the tasks of pose-invariant face recognition (PIFR) and face synthesis across poses. An extensive quantitative and qualitative evaluation carried out on several controlled and in-the-wild benchmarking datasets demonstrates the superiority of the proposed DED-GAN method over the state-of-the-art approaches. Copyright © 2019, The Authors. All rights reserved.

关键词： Generative adversarial networks

Differentiable neural architecture learning for efficient neural network design

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Guo, Qingbei Wu, Xiao-Jun Kittler, Josef Feng, Zhiquan Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Shandong Provincial Key Laboratory of Network based Intelligent Computing University of Jinan Jinan250022 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Automated neural network design has received ever-increasing attention with the evolution of deep convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems that neural architecture search (NAS) confronts is that a large number of candidate neural architectures are required to train, using, for instance, reinforcement learning and evolutionary optimisation algorithms, at a vast computation cost. Even recent differentiable neural architecture search (DNAS) samples a small number of candidate neural architectures based on the probability distribution of learned architecture parameters to select the final neural architecture. To address this computational complexity issue, we introduce a novel architecture parameterisation based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to optimize the neural architecture without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of neural architecture by the continuous scaled sigmoid method with convergence guarantees. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of neural architecture search cost, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageN

关键词： Deep neural networks

Self-grouping convolutional neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

Although group convolution operators are increasingly used in deep convolutional neural networks to improve the computational efficiency and to reduce the number of parameters, most existing methods construct their group convolution architectures by a predefined partitioning of the filters of each convolutional layer into multiple regular filter groups with an equal spatial group size and data-independence, which prevents a full exploitation of their potential. To tackle this issue, we propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN, in which the filters of each convolutional layer group themselves based on the similarity of their importance vectors. Concretely, for each filter, we first evaluate the importance value of their input channels to identify the importance vectors, and then group these vectors by clustering. Using the resulting data-dependent centroids, we prune the less important connections, which implicitly minimizes the accuracy loss of the pruning, thus yielding a set of diverse group convolution filters. Subsequently, we develop two fine-tuning schemes, i.e. (1) both local and global fine-tuning and (2) global only fine-tuning, which experimentally deliver comparable results, to recover the recognition capacity of the pruned network. Comprehensive experiments carried out on the CIFAR-10/100 and ImageNet datasets demonstrate that our self-grouping convolution method adapts to various state-of-the-art CNN architectures, (Figure presented) Figure 1: Evolution of group convolutions. (a) Regular convolution. (b) Regular group convolution. (c) Permuting group convolution. (d) Learned group convolution. (d) Self-grouping convolution. Note that white channels represent the ignored input channels, and gray channels indicate the reused input channels. such as ResNet and DenseNet, and delivers superior performance in terms of compression ratio, speedup and recognition accuracy. We demonstrate the ability of SG-CNN

关键词： Convolution

Contrastive learning based method for X-ray and CT registration under surgical equipment occlusion

学校读者我要写书评

暂无评论

Computers in Biology and Medicine 2024年 180卷 108946页

作者： Wang, Xiyuan Zhang, Zhancheng Xu, Shaokang Luo, Xiaoqing Zhang, Baocheng Wu, Xiao-Jun School of Electronics and Information Engineering at University of Science and Technology Suzhou SuZhou215009 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science at Jiangnan University WuXi214122 China Department of Orthopaedics General Hospital of Central Theater Command of PLA WuHan430012 China Shanghai Jirui Maestro Surgical Technology Co ShangHai200000 China

Deep learning-based 3D/2D surgical navigation registration techniques achieved excellent results. However, these methods are limited by the occlusion of surgical equipment resulting in poor accuracy. We designed a contrastive learning method that treats occluded and unoccluded X-rays as positive samples, maximizing the similarity between the positive samples and reducing interference from occlusion. The designed registration model has Transformer's residual connection (ResTrans), which enhances the long-sequence mapping capability, combined with the contrast learning strategy, ResTrans can adaptively retrieve the valid features in the global range to ensure the performance in the case of occlusion. Further, a learning-based region of interest (RoI) fine-tuning method is designed to refine the misalignment. We conducted experiments on occluded X-rays that contained different surgical devices. The experiment results show that the mean target registration error (mTRE) of ResTrans is 3.25 mm and the running time is 1.59 s. Compared with the state-of-the-art (SOTA) 3D/2D registration methods, our method offers better performance on occluded 3D/2D registration tasks. © 2024 Elsevier Ltd

关键词： Surgical equipment

Learning a representation with the block-diagonal structure for pattern classification

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Yin, He-Feng Wu, Xiao-Jun Kittler, Josef Feng, Zhen-Hua School of Internet of Things Engineering Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

Sparse-representation-based classification (SRC) has been widely studied and developed for various practical signal classification applications. However, the performance of a SRC-based method is degraded when both the training and test data are corrupted. To counteract this problem, we propose an approach that learns Representation with Block-Diagonal Structure (RBDS) for robust image recognition. To be more specific, we first introduce a regularization term that captures the block-diagonal structure of the target representation matrix of the training data. The resulting problem is then solved by an optimizer. Last, based on the learned representation, a simple yet effective linear classifier is used for the classification task. The experimental results obtained on several benchmarking datasets demonstrate the efficacy of the proposed RBDS method. The code will be released upon the acceptance of this paper. Copyright © 2019, The Authors. All rights reserved.

关键词： Image recognition

Multiple riemannian manifold-valued descriptors based image set classification with multi-kernel metric learning

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wang, Rui Wu, Xiao-Jun Kittler, Josef School of Internet of Things Engineering Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

The importance of wild video based image set recognition is becoming monotonically increasing due to the large amount of video resources obtained by diversified video collection approaches, like surveillance, drive recorders, smart phones, and internet. However, the contents of these collected videos are often complicated, and how to efficiently perform set modeling and feature extraction is a big challenge for set-based classification algorithms. In recent years, some proposed image set classification methods have made a considerable advance by modeling the original image set with covariance matrix, linear subspace, or Gaussian distribution. Moreover, the distinctive geometry spanned by them are Symmetric Positive Definite (SPD) manifold, Grassmann manifold, and Gaussian embedded Riemannian manifold, respectively. As a matter of fact, most of them just adopt a single geometric model to describe each given image set, which may lose some other useful information for classification. To tackle this problem, we propose a novel algorithm to model each image set from a multi-geometric perspective. Specifically, the covariance matrix, linear subspace, and Gaussian distribution are applied for set representation simultaneously. In order to fuse these multiple heterogeneous Riemannian manifoldvalued features, the well-equipped Riemannian kernel functions are first utilized to map them into high dimensional Hilbert spaces. Then, a multi-kernel metric learning framework is devised to embed the learned hybrid kernels into a lower dimensional common subspace for classification. We conduct experiments on four widely used datasets corresponding to four different classification tasks: video-based face recognition, set-based object categorization, video-based emotion recognition, and dynamic scene classification, to evaluate the classification performance of the proposed algorithm. Extensive experimental results justify its superiority over the state-of-the-art. Copyright © 2019, Th

关键词： Vector spaces

UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Rao, Dongyu Wu, Xiao-Jun Li, Hui Kittler, Josef Xu, Tianyang Jiangnan University Jiangsu Provincial Engineerinig Laboratory of Pattern Recognition and Computational Intelligence School of Artificial Intelligence and Computer Science Lihu Avenue Wuxi214122 China University of Surrey Centre for Vision Speech and Signal Processing GuildfordGU2 7XH United Kingdom

In this paper, we propose a photorealistic style transfer network to emphasize the natural effect of photo realistic image stylization. In general, distortion of the image content and lacking of details are two typical issues in the style transfer field. To this end, we design a novel framework employing the U-Net structure to maintain the rich spatial clues, with a multi-layer feature aggregation (MFA) method to simultaneously provide the details obtained by the shallow layers in the stylization processing. In particular, an encoder based on the dense block and a decoder form a symmetrical structure of U-Net are jointly staked to realize an effective feature extraction and image reconstruc tion. Besides, a transfer module based on MFA and "adaptive instance normalization" (AdaIN) is inserted in the skip connection positions to achieve the stylization. Accordingly, the stylized image possesses the texture of a real photo and preserves rich content details without introducing any mask or post-processing steps. The experimental results on public datasets demonstrate that our method achieves a more faithful structural similarity with a lower style loss, reflecting the effectiveness and merit of our approach. © 2021, CC0.

关键词： Deep learning

DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Rui Wu, Xiao-Jun Chen, Ziheng Xu, Tianyang Kittler, Josef School of Artificial Intelligence and Computer Science Jiangnan University Wuxi214122 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University China Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom School of Artificial Intelligence and Computer Science Jiangnan University China

Image set-based visual classification methods have achieved remarkable performance, via characterising the image set in terms of a non-singular covariance matrix on a symmetric positive definite (SPD) manifold. To adapt to complicated visual scenarios better, several Riemannian networks (RiemNets) for SPD matrix nonlinear processing have recently been studied. However, it is pertinent to ask, whether greater accuracy gains can be achieved by simply increasing the depth of RiemNets. The answer appears to be negative, as deeper RiemNets tend to lose generalization ability. To explore a possible solution to this issue, we propose a new architecture for SPD matrix learning. Specifically, to enrich the deep representations, we adopt SPDNet [1] as the backbone, with a stacked Riemannian autoencoder (SRAE) built on the tail. The associated reconstruction error term can make the embedding functions of both SRAE and of each RAE an approximate identity mapping, which helps to prevent the degradation of statistical information. We then insert several residual-like blocks with shortcut connections to augment the representational capacity of SRAE, and to simplify the training of a deeper network. The experimental evidence demonstrates that our DreamNet can achieve improved accuracy with increased depth of the network. Copyright © 2022, The Authors. All rights reserved.

关键词： Covariance matrix

Convolutional Neural Network-Based Multimodal Image Information Fusion for Moisture Damage Assessment of Cultural Heritage Buildings

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Wang, Fuzhi Huang, Jizhong Fu, Yu Department of Civil Engineering Shanghai University Shanghai200444 China Institute for the Conservation of Cultural Heritage Shanghai University Shanghai200444 China Key Laboratory of Silicate Cultural Relics Conservation Shanghai University Ministry of Education Shanghai200444 China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University Wuxi214122 China

Water is an essential factor causing the deterioration of historical and cultural heritage, and identifying and quantifying moisture damage is essential for conserving cultural heritage buildings. This study combines infrared thermography (IRT) and microwave technology to investigate moisture-induced diseases and the variation of the spatial distribution of water content in the Yungang Grottoes of China, a World Heritage Site. Information on building humidity distribution was extracted and combined from IRT and microwave information. Furthermore, multi-feature image fusion combines different types of monitoring information through the Convolutional Neural Network (CNN), which is used to diagnose moisture distribution and content accurately. The features obtained from different non-destructive methods were then jointly analyzed using statistical analysis methods such as absolute contrast, noise ratio (SNR), and spatial stability to quantify the artifact moisture defect data, thus improving the accuracy and range of features for detecting water-damaged pathologies. © 2024, The Authors. All rights reserved.

关键词： Convolution