检索结果-内蒙古大学图书馆

UniCanvas: Affordance-Aware Unified Real Image Editing via Customized Text-to-Image Generation

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年第6期133卷 3456-3480页

作者： Jin, Jian Shen, Yang Zhao, Xinyang Fu, Zhenyong Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens Inf PCA LabMinist Educ Nanjing Peoples R China

The demand for assorted conditional edits on a single real image is becoming increasingly prevalent. We focus on two dominant editing tasks that respectively condition on image and text input, namely subject-driven editing and semantic editing. Previous studies typically tackle these two editing tasks separately, thereby demanding multiple editing processes to achieve versatile edits on a single image. However, fragmented and sequential editing processes not only require more user effort but also further degrade the editing quality. In this paper, we propose UniCanvas, an affordance-aware unified framework that can achieve high-quality parallel subject-driven and semantic editing on a single real image within one inference process. UniCanvas innovatively unifies the multimodal inputs of the editing task into the textual condition space using tailored customization strategies. Building upon the unified representations, we propose a novel inference pipeline that performs parallel editing by selectively blending and manipulating two collaborative text-to-image generative branches. Customization enables the editing process to harness the strong visual understanding and reasoning capability of pre-trained generative models for affordance perception, and a unified inference space further facilitates more effective affordance interaction and alignment for compelling editing. Extensive experiments on diverse real images demonstrate that UniCanvas exhibits powerful scene affordance perception in unified image editing, achieving seamless subject-driven editing and precise semantic editing for various target subjects and query prompts (https://***/unicanvas/).

关键词： Real image editing Pre-trained model customization Text-to-image generation Diffusion model Affordance perception

来源：评论

学校读者我要写书评

暂无评论

United Domain Cognition Network for Salient Object Detection in Optical Remote Sensing Images

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2024年 62卷

作者： Sun, Yanguang Yang, Jian Luo, Lei Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens Inf PCA LabMinist Educ Nanjing 210094 Peoples R China

Recently, deep learning-based salient object detection (SOD) in optical remote sensing images (ORSIs) have achieved significant breakthroughs. We observe that existing ORSIs-SOD methods consistently center around optimizing pixel features in the spatial domain, progressively distinguishing between backgrounds and objects. However, pixel information represents local attributes, which are often correlated with their surrounding context. Even with strategies expanding the local region, spatial features remain biased toward local characteristics, lacking the ability of global perception. To address this problem, we introduce the Fourier transform that generate global frequency features and achieve an image-size receptive field. To be specific, we propose a novel united domain cognition network (UDCNet) to jointly explore the global-local information in the frequency and spatial domains. Technically, we first design a frequency-spatial domain transformer (FSDT) block that mutually amalgamates the complementary local spatial and global frequency features to strength the capability of initial input features. Furthermore, a dense semantic excavation (DSE) module is constructed to capture higher level semantic for guiding the positioning of remote sensing objects. Finally, we devise a dual-branch joint optimization (DJO) decoder that applies the saliency and edge branches to generate high-quality representations for predicting salient objects. Experimental results demonstrate the superiority of the proposed UDCNet method over 24 state-of-the-art models, through extensive quantitative and qualitative comparisons in three widely used ORSIs-SOD datasets. The source code is available at: https://***/CSYSI/UDCNet.

关键词： Frequency-domain analysis Image edge detection Semantics Object detection Optical imaging Optical network units Cognition Decoding Optical sensors Remote sensing Fourier transform frequency-spatial information optical remote sensing images (ORSIs) salient object detection (SOD)

来源：评论

学校读者我要写书评

暂无评论

Multiple Resolutions Detail Enhancement Network for Real-Time Image Semantic Segmentation

IEEE Transactions on Artificial Intelligence

引用

IEEE Transactions on Artificial Intelligence 2024年第7期5卷 3393-3407页

作者： Gu, Jing Sun, Xinkai Feng, Jie Yang, Shuyuan Liu, Fang Jiao, Licheng Xidian University Key Lab of Intelligent Perception and Image Understanding of Ministry of Education Xi'an710071 China

Real-time image semantic segmentation (ISS) draws the attentions of more and more researchers as a basis of scene understanding, and it has been applied in many fields that need fast interaction and response, such as autonomous driving and robot control. Considering the loss of low-level spatial information with the deepening network layer, we propose a multiple resolutions detail enhancement network (MRDENet) in this article, which adequately extracts and utilizes accurate low-level detail information from original images with different resolutions. MRDENet consists of three light-weight branch subnetworks, and designs dense oblique connections between adjacent branches to preserve the different level effective features of previous branch. Furthermore, a new multilevel information aggregation (MIA) module is presented to effectively fuse the low-level detail features and the high-level semantic features of different branches by employing group convolution (group conv) and channel shuffle with low computation cost, thus ensuring that MRDENet could achieve a favorable tradeoff between segmentation precision with inference speed. The experimental results show that MRDENet achieves 73.1% mean intersection over union (mIoU) with 93 frames per second (FPS) on Cityscapes dataset, and 68.5% mIoU with 112 FPS on CamVid dataset, which indicates the performance of MRDENet is competitive with the state-of-art methods. © 2020 IEEE.

关键词： Real time systems

来源：评论

学校读者我要写书评

暂无评论

Cascaded Sliding-Window-Based Relativistic GAN Fusion for Perceptual and Consistent Video Super-Resolution 6th

Cascaded Sliding-Window-Based Relativistic GAN Fusion for P...

引用

6th IFIP TC 12 International Conference on Intelligence Science, ICIS 2024

作者： Li, Dingyi PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education and Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China

ISBN: (纸本)9783031712524

Perceptual video super-resolution aims at converting low-resolution videos to visually appealing high-resolution ones. It may lead to temporal inconsistency due to the drastically changing outputs. In this paper, we propose cascaded sliding-window-based relativistic GAN (Generative Adversarial Network) fusion for perceptual and consistent video super-resolution (PC-VSR). Firstly, cascaded sliding-window-based relativistic GAN is designed to extract more useful information. It enlarges the temporal receptive field of sliding-window-based model in each step. It is able to enhance perceptual quality and compensate temporal consistency progressively and sufficiently. The trained separate refinement generator networks are fused into a final refinement generator. The final refinement generator can be calculated recursively at the testing stage. With our generator fusion, the parameter number is reduced and good quality is maintained. Extensive experimental results demonstrate that our approach outperforms state-of-the-art super-resolution methods in terms of perceptual quality. Our method also achieves good temporal consistency and per-pixel accuracy, compared with other perceptual approaches. © IFIP International Federation for Information Processing 2025.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

Global Spatio-Temporal Fusion-based Traffic Prediction Algorithm with Anomaly Aware

Global Spatio-Temporal Fusion-based Traffic Prediction Algor...

引用

2024 IEEE Global Communications Conference, GLOBECOM 2024

作者： Liu, Chaoqun Li, Xuanpeng Gong, Chen Li, Guangyu Nanjing University of Science and Technology Pca Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security China Southeast University Nanjing210094 China

ISBN: (纸本)9798350351255

Traffic prediction is an indispensable component of urban planning and traffic management. Achieving accurate traffic prediction hinges on the ability to capture the potential spatio-temporal relationships among road sensors. However, the majority of existing works focus on local short-term spatiotemporal correlations, failing to fully consider the interactions of different sensors in the long-term state. In addition, these works do not analyze the influences of anomalous factors, or have insufficient ability to extract personalized features of anomalous factors, which make them ineffectively capture their spatiotemporal influences on traffic prediction. To address the aforementioned issues, We propose a global spatio-temporal fusionbased traffic prediction algorithm that incorporates anomaly awareness. Initially, based on the designed anomaly detection network, we construct an efficient anomalous factors impacting module (AFIM), to evaluate the spatio-temporal impact of unexpected external events on traffic prediction. Furthermore, we propose a multi-scale spatio-temporal feature fusion module (MTSFFL) based on the transformer architecture, to obtain all possible both long and short term correlations among different sensors in a wide-area traffic environment for accurate prediction of traffic flow. Finally, experiments are implemented based on real-scenario public transportation datasets (PEMS04 and PEMS08) to demonstrate that our approach can achieve stateof-the-art performance. © 2024 IEEE.

关键词： Weather forecasting

来源：评论

学校读者我要写书评

暂无评论

Quality-aware pattern diffusion for video object segmentation

引用

NEUROCOMPUTING 2023年第1期528卷 148-159页

作者： Zhou, Chuanwei Xu, Chunyan Li, Jun Cui, Zhen Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Social PCA LabKey Lab Intelligent Percept & Syst High Di Nanjing Peoples R China

Recently, great progresses have been achieved under the support of memory mechanism when dealing with video object segmentation (VOS) problem. Despite its achievements, existing VOS approaches still suffer from abnormal samples, which derive from intrinsic video artifacts such as occlusion and motion blur. To mitigate the above issue, in this paper, we propose a quality-aware pattern diffusion (QPD) framework to boost the VOS performance. To achieve quality-aware pattern diffusion, a quality align-ment mechanism is proposed, and it aims to promote the contributions of those normal samples while suppressing those abnormal ones during the feature propagation/diffusion processes. With our proposed quality alignment mechanism, the diffused instance features could be kept staying in the normal feature space, keeping from feature contamination caused by those low-quality samples. We first introduce a learnable quality evaluator to assess the sample qualities in both the temporal domain (i.e., across the historical frames), as well as the spatial domain (i.e., within the current frame). To achieve adaptive his-torical feature propagation into the current instance, a quality-aware long-term context propagation module is proposed, with which more stable instance representations could be achieved through the established quality-aware feature propagation process. A quality-aware pattern diffusion module is fur-ther introduced to address the spatial-domain abnormal samples, resulting in effective decoder feature refinement through building the quality-aware correspondence weights. Extensive experiments have demonstrated that our proposed quality alignment mechanism could boost the performance by a great margin over a strong baseline while achieving state-of-the-art performances on public VOS benchmarks.(c) 2023 Elsevier B.V. All rights reserved.

关键词： Video object segmentation Quality -aware pattern diffusion Quality alignment mechanism

来源：评论

学校读者我要写书评

暂无评论

RagNet3D: Learning distinguishable representation for pooled grids in 3D object detection

引用

NEUROCOMPUTING 2025年 635卷

作者： Chen, Jiaxin Han, Yuehui Yan, Zhiqiang Qian, Jianjun Li, Jun Yang, Jian Nanjing Univ Sci & Technol PCA Lab Key Lab Intelligent Percept & Syst High Dimens Inf Minist Educ Nanjing Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Social Nanjing Peoples R China

3D object detection plays a crucial role in autonomous driving. Previous two-stage 3D detectors have developed Grid-based Region-of-Interest (RoI) Pooling techniques, such as RoI-Grid Pooling and Voxel Pooling, both of which quantify RoI into grids. However, the grids are usually ambiguous since parts of them pertain to multiple RoIs that identify a single object. To address this issue, we propose a RoI-Aware Grids Pooling Network (RagNet3D), which introduces RoI-View context to create distinguishable grid representations. Specifically, we present a RoI-View Prediction module that predicts RoI-View context via the guidance of the distance between RoIs and objects. Meanwhile, we propose a Couple-View Fusion module that propagates the probabilistic distribution, calculated from the RoI-View context, into the RoI-irrelevant grid features for further box refinement. Extensive experiments on KITTI and Waymo Open Dataset show that our method achieves remarkable improvements against the baselines.

关键词： 3D object detection Grid based pooling RoI-View context

来源：评论

学校读者我要写书评

暂无评论

Graph Representation Learning Meets Computer Vision: A Survey

IEEE Transactions on Artificial Intelligence

引用

IEEE Transactions on Artificial Intelligence 2023年第1期4卷 2-22页

作者： Jiao, Licheng Chen, Jie Liu, Fang Yang, Shuyuan You, Chao Liu, Xu Li, Lingling Hou, Biao Key Lab. of Intelligent Perception and Image Understanding of the Ministry of Education of China International Research Center of Intelligent Perception and Computation School of Artificial Intelligence Xidian University Xi'an710071 China

A graph structure is a powerful mathematical abstraction, which can not only represent information about individuals but also capture the interactions between individuals for reasoning. Geometric modeling and relational inference based on graph data is a long-standing topic of interest in the computer vision community. In this article, we provide a systematic review of graph representation learning and its applications in computer vision. First, we sort out the evolution of representation learning on graphs, categorizing them into the nonneural network and neural network methods based on the way the nodes are encoded. Specifically, nonneural network methods, such as graph embedding and probabilistic graphical models, are introduced, and neural network methods, such as graph recurrent neural networks, graph convolutional networks, and variants of graph neural networks, are also presented. Then, we organize the applications of graph representation algorithms in various vision tasks (such as image classification, semantic segmentation, object detection, and tracking) for review and reference, and the typical graph construction approaches in computer vision are also summarized. Finally, on the background of biology and brain inspiration, we discuss the existing challenges and future directions of graph representation learning and computer vision. © 2022 IEEE.

关键词： Computer vision

来源：评论

学校读者我要写书评

暂无评论

Hierarchical keypoints Feature Alignment for Domain Adaptive Pose Estimation

引用

NEUROCOMPUTING 2025年 611卷

作者： Xu, Jie Liu, Yunan Yang, Jian Zhang, Shanshan Nanjing Univ Sci & Technol Minist Educ PCA Lab Key Lab Intelligent Percept & Syst High Dimens Inf Nanjing 210094 Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Social Nanjing 210094 Peoples R China

Unsupervised Domain Adaptation (UDA) is a popular research topic in computer vision. One of the most common strategies in this field is to reduce domain shifts through global feature alignment. Unfortunately, we observe its failure on adaptive human pose estimation. From our analysis, we find out two major reasons: the extreme imbalance between keypoints and non-keypoints regions and the high diversity of human skeleton structures. To address these problems, we propose a simple yet effective approach named PoseDA. Our idea is to let the alignment focus on the feature of keypoint regions and enforce the model to learn domain-invariant representations for keypoint prediction. The key component of our PoseDA is a Hierarchical Feature Selection of keypoints Regions (HFS) module, which consists of Coarse Feature Selection of keypoints Regions (CFS) and Reliable Feature Selection of keypoints Regions (RFS). By using HFS, we obtain reliable and compact keypoints features, allowing our model to achieve more effective feature alignment. Under three UDA scenarios, i.e. JTA -> MPII, SURREAL -> LSP, SURREAL -> Human3.6, our PoseDA establishes new state-of-the-art performance. In particular, our PoseDA outperforms the previous best UDA methods by over 7% w.r.t. PCKh on JTA -> MPII.

关键词： Human pose estimation Unsupervised domain adaptation Feature alignment

来源：评论

学校读者我要写书评

暂无评论

Progressive Exploration-Conformal Learning for Sparsely Annotated Object Detection in Aerial Images 38

Progressive Exploration-Conformal Learning for Sparsely Anno...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Lu, Zihan Wang, Chenxu Xu, Chunyan Zheng, Xiangwei Cui, Zhen PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China Shandong Normal University Jinan China

The ability to detect aerial objects with limited annotation is pivotal to the development of real-world aerial intelligence systems. In this work, we focus on a demanding but practical sparsely annotated object detection (SAOD) in aerial images, which encompasses a wider variety of aerial scenes with the same number of annotated objects. Although most existing SAOD methods rely on fixed thresholding to filter pseudo-labels for enhancing detector performance, adapting to aerial objects proves challenging due to the imbalanced probabilities/confidences associated with predicted aerial objects. To address this problem, we propose a novel Progressive Exploration-Conformal Learning (PECL) framework to address the SAOD task, which can adaptively perform the selection of high-quality pseudo-labels in aerial images. Specifically, the pseudo-label exploration can be formulated as a decision-making paradigm by adopting a conformal pseudo-label explorer and a multi-clue selection evaluator. The conformal pseudo-label explorer learns an adaptive policy by maximizing the cumulative reward, which can decide how to select these high-quality candidates by leveraging their essential characteristics and inter-instance contextual information. The multi-clue selection evaluator is designed to evaluate the explorer-guided pseudo-label selections by providing an instructive feedback for policy optimization. Finally, the explored pseudo-labels can be adopted to guide the optimization of aerial object detector in a closed-loop progressive fashion. Comprehensive evaluations on two public datasets demonstrate the superiority of our PECL when compared with other state-of-the-art methods in the sparsely annotated aerial object detection task. The code will be available at: https://***/SAOD-research/PECL. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：