检索结果-内蒙古大学图书馆

Learning to Hallucinate Face in the Dark

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 2314-2326页

作者： Wang, Yuanzhi Lu, Tao Yao, Yuan Zhang, Yanduo Xiong, Zixiang Wuhan Inst Technol Hubei Key Lab Intelligent Robot Wuhan 430205 Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Key Lab Intelligent Percept & Syst High Dimens Inf Nanjing 210094 Peoples R China Hubei Univ Arts & Sci Comp Sch Xiangyang 441053 Peoples R China Texas A&M Univ Dept Elect & Comp Engn College Stn TX 77843 USA

Face hallucination in low-light environments is an extremely challenging task due to the significant loss of facial structure and facial texture information. Although cascading image relighting and face hallucination tasks is a feasible strategy, simply cascading these two tasks does not achieve satisfactory results because they do not fit into each other naturally. In this article, we propose a novel duplex fusing-embedding learning approach to tackle this challenge in low-light environments. The core of the proposed approach is the duplexity of feature fusion and embedding between relighting and hallucination tasks. In the feature fusion phase, the shallow features from two tasks are bidirectionally fused and activated into a consistent feature space. In the feature embedding phase, the fused features from the previous iteration are fed back and bidirectionally embedded into the deep features of two tasks in the current iteration so that they can learn feature representations that consistently represent both tasks, thereby boosting the performance of relighting and hallucination to generate photorealistic HR face images. Experimental results show that the proposed approach allows current face hallucination methods to learn to hallucinate face in the dark.

关键词： Face hallucination image relighting duplex fusing-embedding learning feature fusion feature embedding

来源：评论

学校读者我要写书评

暂无评论

Structure Flow-Guided Network for Real Depth Super-resolution 37

Structure Flow-Guided Network for Real Depth Super-resolutio...

引用

37th AAAI Conference on Artificial Intelligence (AAAI) / 35th Conference on Innovative Applications of Artificial Intelligence / 13th Symposium on Educational Advances in Artificial Intelligence

作者： Yuan, Jiayi Jiang, Haobo Li, Xiang Qian, Jianjun Li, Jun Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia PCA LabKey Lab Intelligent Percept & Syst HighDi Nanjing Peoples R China

ISBN: (纸本)9781577358800

Real depth super-resolution (DSR), unlike synthetic settings, is a challenging task due to the structural distortion and the edge noise caused by the natural degradation in real-world low-resolution (LR) depth maps. These defeats result in significant structure inconsistency between the depth map and the RGB guidance, which potentially confuses the RGB-structure guidance and thereby degrades the DSR quality. In this paper, we propose a novel structure flow-guided DSR framework, where a cross-modality flow map is learned to guide the RGB-structure information transferring for precise depth upsampling. Specifically, our framework consists of a cross-modality flow-guided upsampling network (CFU-Net) and a flow-enhanced pyramid edge attention network (PEANet). CFUNet contains a trilateral self-attention module combining both the geometric and semantic correlations for reliable cross-modality flow learning. Then, the learned flow maps are combined with the grid-sampling mechanism for coarse high-resolution (HR) depth prediction. PEANet targets at integrating the learned flow map as the edge attention into a pyramid network to hierarchically learn the edge-focused guidance feature for depth edge refinement. Extensive experiments on real and synthetic DSR datasets verify that our approach achieves excellent performance compared to state-of-the-art methods. Our code is available at: https://***/Yuanjiayii/DSR SFG.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Few-shot Continual Infomax Learning

Few-shot Continual Infomax Learning

引用

IEEE/CVF International Conference on Computer Vision (ICCV)

作者： Gu, Ziqi Xu, Chunyan Yang, Jian Cui, Zhen Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Key Lab Intelligent Percept & Syst High Dimens In Nanjing Peoples R China

ISBN: (纸本)9798350307184

Few-shot continual learning is the ability to continually train a neural network from a sequential stream of few-shot data. In this paper, we propose a Few-shot Continual Infomax Learning (FCIL) framework that makes a deep model to continually/incrementally learn new concepts from few labeled samples, relieving the catastrophic forgetting of past knowledge. Specifically, inspired by the theoretical definition of transfer entropy, we introduce a feature embedding infomax to effectively perform the few-shot learning, which can transfer the strong encoding capability of the base network to learn the feature embedding of these novel classes by maximizing the mutual information of different-level feature distributions. Further, considering that the learned knowledge in the human brain is a generalization of actual information and exists in a certain relational structure, we perform continual structure infomax learning to relieve the catastrophic forgetting problem in the continual learning process. The information structure of this learned knowledge can be preserved through maximizing the mutual information across these continual-changing relations of inter- classes. Comprehensive evaluations on CIFAR100, miniImageNet, and CUB200 datasets demonstrate the superiority of our FCIL when compared against state-of-the-art methods on the few-shot continual learning task.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Instance-Aware Deep Graph Learning for Multi-label Classification

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2023年 25卷 90-99页

作者： Wang, Yun Zhang, Tong Zhou, Chuanwei Cui, Zhen Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Key Lab Intelligent Percept & Syst High Dimens Inf Nanjing 210094 Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Social Nanjing 210094 Peoples R China

Graph convolutional neural network (GCN) has effectively boosted the multi-label image recognition task by modeling correlation among labels. In previous methods, label correlation is computed based on statistical information through label diffusion, and therefore the same for all samples. This, however, makes graph inference on labels insufficient to handle huge variations among numerous image instances. In this paper, we propose an instance-aware graph convolutional neural network (IA_GCN) framework for the multi-label classification. As a whole, two fused branches of sub-networks are involved in the framework: a global branch modeling the whole image and a local branch exploring dependencies among regions of interests (ROIs). For both the branches, an image-dependent label correlation matrix (ID_LCM), fusing both the statistical label correlation matrix (LCM) and an individual one of each image instance, is constructed to inject adaptive information of label-awareness into the learned features of the model through graph convolution. Specifically, the individual LCM of each image is obtained by mining the label dependencies based on the predicted label scores of those detected ROIs. In this process, considering the contribution differences of ROIs to multi-label classification, variational inference is introduced to learn adaptive scaling factors for those ROIs by considering their complex distribution. Finally, extensive experiments on MS-COCO and VOC datasets show that our proposed approach outperforms existing state-of-the-art methods.

关键词： Correlation Adaptation models Task analysis Feature extraction Image recognition Convolutional neural networks Sports Graph convolutional neural network image-dependent label correlation matrix regions of interests variational inference

来源：评论

学校读者我要写书评

暂无评论

Exploratory Inference Learning for Scribble Supervised Semantic Segmentation 37

Exploratory Inference Learning for Scribble Supervised Seman...

引用

37th AAAI Conference on Artificial Intelligence (AAAI) / 35th Conference on Innovative Applications of Artificial Intelligence / 13th Symposium on Educational Advances in Artificial Intelligence

作者： Zhou, Chunawei Cui, Zhen Xu, Chunyan Han, Cao Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Key Lab Intelligent Percept & Syst High Dimens In Nanjing Peoples R China

ISBN: (纸本)9781577358800

Scribble supervised semantic segmentation has achieved great advances in pseudo label exploitation, yet suffers insufficient label exploration for the mass of unannotated regions. In this work, we propose a novel exploratory inference learning (EIL) framework, which facilitates efficient probing on unlabeled pixels and promotes selecting confident candidates for boosting the evolved segmentation. The exploration of unannotated regions is formulated as an iterative decision-making process, where a policy searcher learns to infer in the unknown space and the reward to the exploratory policy is based on a contrastive measurement of candidates. In particular, we devise the contrastive reward with the intra-class attraction and the inter-class repulsion in the feature space w.r. t the pseudo labels. The unlabeled exploration and the labeled exploitation are jointly balanced to improve the segmentation, and framed in a close-looping end-to-end network. Comprehensive evaluations on the benchmark datasets (PASCAL VOC 2012 and PASCAL Context) demonstrate the superiority of our proposed EIL when compared with other state-of-the-art methods for the scribble-supervised semantic segmentation problem.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Recurrent Structure Attention Guidance for Depth Super-resolution 37

Recurrent Structure Attention Guidance for Depth Super-resol...

引用

37th AAAI Conference on Artificial Intelligence (AAAI) / 35th Conference on Innovative Applications of Artificial Intelligence / 13th Symposium on Educational Advances in Artificial Intelligence

作者： Yuan, Jiayi Jiang, Haobo Li, Xiang Qian, Jianjun Li, Jun Yang, Jian Nanjing Univ Sci & Technol PCA LabKey Lab Intelligent Percept & Syst HighDi Minist EducJiangsu Key Lab Image & Video Underst Sch Comp Sci & Engn Nanjing Peoples R China

ISBN: (纸本)9781577358800

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) frame-work, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods. Our code is available at: https://***/Yuanjiayii/DSR RSAG.

关键词： Frequency domain analysis

来源：评论

学校读者我要写书评

暂无评论

Incomplete Multimodality-Diffused Emotion Recognition 37

Incomplete Multimodality-Diffused Emotion Recognition

引用

37th Conference on Neural Information Processing Systems (NeurIPS)

作者： Wang, Yuanzhi Li, Yong Cui, Zhen Nanjing Univ Sci & Technol Sch Comp Sci & Engn PCA Lab Minist EducKey Lab Intelligent Percept & Syst Hi Nanjing Peoples R China

ISBN: (纸本)9781713899921

Human multimodal emotion recognition (MER) aims to perceive and understand human emotions via various heterogeneous modalities, such as language, vision, and acoustic. Compared with unimodality, the complementary information in the multimodalities facilitates robust emotion understanding. Nevertheless, in real-world scenarios, the missing modalities hinder multimodal understanding and result in degraded MER performance. In this paper, we propose an Incomplete Multimodality-Diffused emotion recognition (IMDer) method to mitigate the challenge of MER under incomplete multimodalities. To recover the missing modalities, IMDer exploits the score-based diffusion model that maps the input Gaussian noise into the desired distribution space of the missing modalities and recovers missing data abided by their original distributions. Specially, to reduce semantic ambiguity between the missing and the recovered modalities, the available modalities are embedded as the condition to guide and refine the diffusion-based recovering process. In contrast to previous work, the diffusion-based modality recovery mechanism in IMDer allows to simultaneously reach both distribution consistency and semantic disambiguation. Feature visualization of the recovered modalities illustrates the consistent modality-specific distribution and semantic alignment. Besides, quantitative experimental results verify that IMDer obtains state-of-the-art MER accuracy under various missing modality patterns. Codes are released at https://***/mdswyz/IMDer.

关键词： Speech recognition

来源：评论

学校读者我要写书评

暂无评论

Distribution-Consistent Modal Recovering for Incomplete Multimodal Learning

Distribution-Consistent Modal Recovering for Incomplete Mult...

引用

IEEE/CVF International Conference on Computer Vision (ICCV)

作者： Wang, Yuanzhi Cui, Zhen Li, Yong Nanjing Univ Sci & Technol Minist Educ Sch Comp Sci & Engn PCA LabKey Lab Intelligent Percept & Syst High D Nanjing Peoples R China

ISBN: (纸本)9798350307184

Recovering missing modality is popular in incomplete multimodal learning because it usually benefits downstream tasks. However, the existing methods often directly estimate missing modalities from the observed ones by deep neural networks, lacking consideration of the distribution gap between modalities, resulting in the inconsistency of distributions between the recovered and the true data. To mitigate this issue, in this work, we propose a novel recovery paradigm, Distribution-Consistent Modal Recovering ( DiCMoR), to transfer the distributions from available modalities to missing modalities, which thus maintains the distribution consistency of recovered data. In particular, we design a class- specific flow based modality recovery method to transform cross-modal distributions on the condition of sample class, which could well predict a distributionconsistent space for missing modality by virtue of the invertibility and exact density estimation of normalizing flow. The generated data from the predicted distribution is integrated with available modalities for the task of classification. Experiments show that DiCMoR gains superior performances and is more robust than existing state-of-the-art methods under various missing patterns. Visualization results show that the distribution gaps between recovered modalities and missing modalities are mitigated. Codes are released at https://***/mdswyz/DiCMoR.

关键词：

来源：评论

学校读者我要写书评

暂无评论

SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation 38

SHaRPose: Sparse High-Resolution Representation for Human Po...

引用

38th AAAI Conference on Artificial Intelligence (AAAI) / 36th Conference on Innovative Applications of Artificial Intelligence / 14th Symposium on Educational Advances in Artificial Intelligence

作者： An, Xiaoqi Zhao, Lin Gong, Chen Wang, Nannan Wang, Di Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia PCA LabKey Lab Intelligent Percept & Syst High D Nanjing Peoples R China Xidian Univ State Key Lab Integrated Serv Networks Xian Peoples R China

ISBN: (纸本)1577358872

High-resolution representation is essential for achieving good performance in human pose estimation models. To obtain such features, existing works utilize high-resolution input images or fine-grained image tokens. However, this dense high-resolution representation brings a significant computational burden. In this paper, we address the following question: "Only sparse human keypoint locations are detected for human pose estimation, is it really necessary to describe the whole image in a dense, high-resolution manner?" Based on dynamic transformer models, we propose a framework that only uses Sparse High-resolution Representations for human Pose estimation (SHaRPose). In detail, SHaRPose consists of two stages. At the coarse stage, the relations between image regions and keypoints are dynamically mined while a coarse estimation is generated. Then, a quality predictor is applied to decide whether the coarse estimation results should be refined. At the fine stage, SHaRPose builds sparse high-resolution representations only on the regions related to the keypoints and provides refined high-precision human pose estimations. Extensive experiments demonstrate the outstanding performance of the proposed method. Specifically, compared to the state-of-the-art method ViTPose, our model SHaRPose-Base achieves 77.4 AP (+0.5 AP) on the COCO validation set and 76.7 AP (+0.5 AP) on the COCO test-dev set, and infers at a speed of 1.4x faster than ViTPose-Base. Code is available at https://***/AnxQ/sharpose.

关键词： Artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

Uncertainty-aware Superpoint Graph Transformer for Weakly Supervised 3D Semantic Segmentation

引用

IEEE Transactions on Fuzzy Systems 2025年第6期33卷 1899-1912页

作者： Fan, Yan Wang, Yu Zhu, Pengfei Hui, Le Xie, Jin Hu, Qinghua Tianjin University College of Intelligence and Computing Tianjin300350 China Ministry of Education of the People's Republic of China Engineering Research Center of City Intelligence and Digital Governance China Haihe Laboratory of Itai Tianjin China Northwestern Polytechnical University School of Electronics and Information Shaanxi Key Laboratory of Information Acquisition and Processing China Nanjing University of Science and Technology Pca Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Nanjing University State Key Laboratory for Novel Software Technology Nanjing China

Weakly supervised 3D semantic segmentation has successfully mitigated the labor-intensive and time-consuming task of annotating 3D point clouds. However, reliably utilizing the minimal point-wise annotations for unlabeled data in complex and large-scale scenes is still challenging, such as only 20 points labeled in 2 million points. To tackle this challenge, we propose a new Uncertainty-aware Superpoint Graph Transformer (UaSGT) framework that utilizes minimal annotations for unlabeled data learning through reliable long-range supervision propagation from labeled superpoints to unlabeled superpoints. First, we propose a superpoint graph transformer to achieve long-range supervision propagation along the attention-based fuzzy subsets defined on superpoints. The attention-based fuzzy subset measures the membership of unlabeled superpoints to clusters centered on labeled superpoints. Second, we employ an uncertainty-aware membership rectification technique on the fuzzy subset to ensure reliable propagation among superpoints within the same category. This technique integrates an uncertainty prediction module to mask the influence of unreliable membership and a spatial prior refinement module to reduce uncertainty in intra-class membership degrees. Finally, experimental results on two large-scale benchmarks S3DIS and ScanNet-V2 demonstrate the superiority of our approach compared to the state-of-the-art with at least 90% annotation reduction, and our method also achieves comparable performance to fully supervised methods with less than 0.1% labeled points. Our code is available at https://***/fanyan0411/UaSGT. © 2025 IEEE.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：