检索结果-内蒙古大学图书馆

scene graph generation based on lightweight entity pair object detection and relation classification ensemble

NEUROCOMPUTING 2025年 637卷

作者： Hu, Hong-Xiang Yang, Xu-Hua Zhao, Yu-Yong Zhejiang Univ Technol Coll Comp Sci & Technol Hangzhou 310023 Peoples R China

scene graph generation (SGG) aims to automatically generate a semantic graph structure, enabling a deeper understanding and reasoning of visual scenes. It is widely used in scenarios such as autonomous driving, virtual reality and smart cities. Existing SGG models generally have problems such as complex structure, large number of parameters, and difficulty in deployment. At the same time, the bias in predicate relationship prediction caused by the long-tail distribution of training data affects the overall performance of SGG. In order to improve the limitations of the above methods, first, we propose a lightweight entity pair object detection method that can directly and efficiently decode subject-object pairs that may have predicate relations from images. Next, in order to alleviate the long-tail distribution, we propose a predicate relation classifier optimization method (PRCE) based on ensemble learning. The original training set is used to generate multiple new training sets based on varied sampling rules, and distinct predicate relationship classifiers are trained with the resulting classifiers integrated for final predicate relationship prediction. On the Visual Genome (VG) and OpenImageV6 datasets, by comparing with well-known methods, experiments show that our proposed model achieves very competitive performance. The code is available at: https://***/xhonghu/match-prce.

关键词： scene graph generation Lightweight object detection Ensemble methods Long-tail distribution

来源：评论

学校读者我要写书评

暂无评论

scene Adaptive Context Modeling and Balanced Relation Prediction for scene graph generation

引用

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 2025年第3期21卷 1-19页

作者： Xu, Kai Wang, Lichun Li, Shuang Gao, Tong Yin, Baocai Beijing Univ Technol Beijing Key Lab Multimedia & Intelligent Software Beijing Peoples R China Beijing Informat Sci & Technol Univ Sch Automat Beijing Peoples R China

scene graph generation (SGG) aims to perceive objects and their relations in images, which can bridge the gap between upstream detection tasks and downstream high-level visual understanding tasks. For SGG models, over-fitting head predicates can lead to bias in the generated scene graph, which has become a consensus. A series of debiasing methods have been proposed to solve the problem. However, some existing debiasing SGG methods have a tendency to over-fit tail predicates, which is another type of bias. In order to eliminate the one-way over-fitting of head or tail predicates, this article proposes a balanced relation prediction (BRP) module which is model-agnostic and compatible with existing re-balancing methods. Moreover, because the relation prediction is based on object feature representation, this article proposes a scene adaptive context fusion (SACF) module to refine the object feature representation. Specifically, SACF models the context based on a chain structure, where the order of objects in the chain structure is adaptively arranged according to the scene content, achieving visual information fusion that adapts to the scene where the objects are located. Experiments on VG and GQA datasets show that the proposed method achieves competitive results on the comprehensive metric of R@K and mR@K.

关键词： scene graph generation Deep Network scene Adaptive Context Modeling Balanced Relation Prediction

来源：评论

学校读者我要写书评

暂无评论

Attention redirection transformer with semantic oriented learning for unbiased scene graph generation

引用

PATTERN RECOGNITION 2025年 158卷

作者： Zhang, Ruonan An, Gaoyun Cen, Yigang Ruan, Qiuqi Beijing Jiaotong Univ Inst Informat Sci Beijing Peoples R China Beijing Key Lab Adv Informat Sci & Network Technol Beijing Peoples R China

scene graph generation (SGG) plays an important role in scene understanding because all of the objects and relations in an image can be abstracted into a concise topological graph. Due to the complexity of visual scenes, including mutual occlusion between objects and semantic ambiguity, SGG is still a challenging task. Most of the existing models only focused on the context of a single object while contexts provided by paired objects are ignored. In this paper, we propose an Attention Redirection Transformer (ART) to extract pair-level contexts specifically, which is divided into an attention distraction stage and an attention integration stage. In this way, the attention of the model is forced to be redirected, which explores the implicit information in the background. In addition, to incorporate the semantic information of predicates, a Semantic Oriented Learning Module (SOL) is designed, which may assist in getting better textual semantics and also prompts cross-modal information fusion. At last, a self-diversity driven Dual Translation Embedding Module (DTM) is designed, which refines representations of subject and object and makes them distinct. Experimental results on the Visual Genome dataset demonstrate the effectiveness of our proposed method. Moreover, our method outperforms state-of-the-art methods on the mR@K metric. The source codes are released on Github: https://***/Nora-Zhang98/ART-SOL.

关键词： scene graph generation Transformer Attention redirection Translation embedding scene understanding

来源：评论

学校读者我要写书评

暂无评论

Relation-Specific Feature Augmentation for unbiased scene graph generation

引用

PATTERN RECOGNITION 2025年 157卷

作者： Liu, Zhihong Wang, Jianji Chen, Hui Ma, Yongqiang Zheng, Nanning Xi An Jiao Tong Univ Inst Artificial Intelligence & Robot Natl Engn Res Ctr Visual Informat & Applicat Natl Key Lab Human Machine Hybrid Augmented Intell Xian 710049 Shanxi Peoples R China

scene graph generation (SGG) models suffer from the long-tailed distribution of relations, which results in biased predictions that favor head relations (e.g. , on) over informative tail ones (e.g. , sitting on, laying on, standing on). Existing solutions typically adopt class re-balancing strategies to balance data distribution. However, they do not essentially solve the lack of information due to insufficient tail data. To this end, we propose a Relation-Specific Feature Augmentation (RSFA) framework to mitigate the longtailed bias by augmenting relations in the feature space. To perform augmentation effectively, we design an augmentation scheme and a novel Dual Attention Network (DAN). The augmentation scheme augments each relation uniformly based on the reciprocal number of samples to avoid over-fitting. By extracting relation- specific information from new object features generated by a Conditional Variational AutoEncoder (CVAE), DAN generates reliable virtual relation representations to provide useful information to guide optimizing relation classifier. Extensive ablation studies and comprehensive analysis demonstrate the effectiveness of our method in debiasing. And results on the Visual Genome benchmark show that our method significantly outperforms the existing state-of-the-art methods.

关键词： Image understanding scene graph generation Long-tailed distribution Feature augmentation

来源：评论

学校读者我要写书评

暂无评论

Relationship-Incremental scene graph generation by a Divide-and-Conquer Pipeline With Feature Adapter

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2025年 34卷 678-688页

作者： Li, Xuewei Zheng, Guangcong Yu, Yunlong Ji, Naye Li, Xi Zhejiang Univ Coll Comp Sci & Technol Hangzhou 310027 Peoples R China Zhejiang Prov Key Lab Informat Proc Commun & Netwo Hangzhou 310007 Peoples R China Commun Univ Zhejiang Coll Media Engn Hangzhou 310018 Peoples R China Zhejiang Singapore Innovat & AI Joint Res Lab Hangzhou 310027 Peoples R China

As a challenging computer vision task, scene graph generation (SGG) finds the latent semantic relationships among objects from a given image, which may be limited by the datasets and real-world scenarios. In this paper, we consider a novel incremental learning task called Relationship-Incremental scene graph generation (RISGG) that learns the semantic relationships among objects in an incremental way. Compared with classic Class-Incremental Learning (CIL) problem, RISGG suffers from its special issues: 1) Old class shift - the relationship-labeled object pair may have different labels during different learning sessions;2) Background shift - the relationship-unlabeled object pair may not be a real unlabeled one. In this work, we address the above issues from the following aspects. First, we present a Divide-and-Conquer (DaC) pipeline to deal with the old class shift via decoupling the recognition of relationship classes and recognizing relationships individually. In this way, label confusion and interaction among different relationships are eliminated during training. Second, we propose a Feature Adapter (FA) to bridge the feature space gap between the current session and the previous one and use our extra supervision to mine old relationship information in the current session. Our proposed network combined DaC and FA, abbreviated DaCFA-Net, for RISGG. Experimental results on the benchmark dataset demonstrate the significant performance gain of DaCFA-Net in RISGG. It gains about 20% improvement against the SGG baselines on the popular VG dataset.

关键词： Task analysis Training Wheels Dogs Pipelines Feature extraction Detectors Class-incremental learning scene graph generation feature adapter divide-and-conquer pipeline

来源：评论

学校读者我要写书评

暂无评论

Informative scene graph generation via Debiasing

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年 1-24页

作者： Gao, Lianli Lyu, Xinyu Guo, Yuyu Hu, Yuxuan Li, Yuan-Fang Xu, Lu Shen, Heng Tao Song, Jingkuan Univ Elect Sci & Technol China Shenzhen Inst Adv Study Shenzhen Peoples R China Southwestern Univ Finance & Econ Chengdu Peoples R China Southwest Univ Chongqing Peoples R China Monash Univ Melbourne Vic Australia Kuaishou Beijing Peoples R China Tongji Univ Shanghai Peoples R China

scene graph generation aims to detect visual relationship triplets, (subject, predicate, object). Due to biases in data, current models tend to predict common predicates, e.g., "on" and "at", instead of informative ones, e.g., "standing on" and "looking at". This tendency results in the loss of precise information and overall performance. If a model only uses "stone on road" rather than "stone blocking road" to describe an image, it may be a grave misunderstanding. We argue that this phenomenon is caused by two imbalances: semantic space level imbalance and training sample level imbalance. For this problem, we propose DB-SGG, an effective framework based on debiasing but not the conventional distribution fitting. It integrates two components: Semantic Debiasing (SD) and Balanced Predicate Learning (BPL), for these imbalances. SD utilizes a confusion matrix and a bipartite graph to construct predicate relationships. BPL adopts a random undersampling strategy and an ambiguity removing strategy to focus on informative predicates. Benefiting from the model-agnostic process, our method can be easily applied to SGG models and outperforms Transformer by 136.3%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$136.3\%$$\end{document}, 119.5%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$119.5\%$$\end{document}, and 122.6%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$122.6\%$$\end{document} on mR@20 at three SGG sub-tasks on the SGG-VG dat

关键词： scene graph generation Visual relationship Debaising Information content.

来源：评论

学校读者我要写书评

暂无评论

A Causal Adjustment Module for Debiasing scene graph generation

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2025年第5期47卷 4024-4043页

作者： Liu, Li Sun, Shuzhou Zhi, Shuaifeng Shi, Fan Liu, Zhen Heikkila, Janne Liu, Yongxiang NUDT Coll Elect Sci & Technol Changsha 41007 Hunan Peoples R China Tsinghua Univ Dept Comp Sci & Technol Beijing 100190 Peoples R China Univ Oulu Ctr Machine Vis & Signal Anal CMVS Oulu 90570 Finland NUDT Coll Elect Engn Hefei 41007 Hunan Peoples R China

While recent debiasing methods for scene graph generation (SGG) have shown impressive performance, these efforts often attribute model bias solely to the long-tail distribution of relationships, overlooking the more profound causes stemming from skewed object and object pair distributions. In this paper, we employ causal inference techniques to model the causality among these observed skewed distributions. Our insight lies in the ability of causal inference to capture the unobservable causal effects between complex distributions, which is crucial for tracing the roots of model bias. Specifically, we introduce the Mediator-based Causal Chain Model (MCCM), which, in addition to modeling causality among objects, object pairs, and relationships, incorporates mediator variables, i.e., cooccurrence distribution, for complementing the causality. Following this, we propose the Causal Adjustment Module (CAModule) to estimate the modeled causal structure, using variables from MCCM as inputs to produce a set of adjustment factors aimed at correcting biased model predictions. Moreover, our method enables the composition of zero-shot relationships, thereby enhancing the model's ability to recognize such relationships. Experiments conducted across various SGG backbones and popular benchmarks demonstrate that CAModule achieves state-of-the-art mean recall rates, with significant improvements also observed on the challenging zero-shot recall rate metric.

关键词： Tail Heavily-tailed distribution Visualization Predictive models Magnetic heads Cause effect analysis Measurement Accuracy Training Computational modeling Causal inference longtailed distribution scene graph generation

来源：评论

学校读者我要写书评

暂无评论

Union-Redefined Prototype Network for scene graph generation

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 280卷

作者： Jung, Namgyu Choi, Chang Gachon Univ 1342 Seongnam Daero Seongnam 13120 Gyeonggi Do South Korea

Recent advances in scene graph generation employ commonsense knowledge to model visual prototypes for various predicates. However, indiscriminate application of prototypes to all entity pairs within a predicate can lead to confusion when interpreting the same entity pairs that share similar visual features but correspond to different predicates. Particularly, clearly distinguishing predicates often requires careful consideration of subtle factors, such as gaze direction, spatial distance, and surrounding context. While these aspects are inherently part of the non-overlapping regions between the subject and object, they are frequently overlooked in practice. In this paper, we propose the Union-Redefined Prototype Network (UP-Net), which effectively captures predicate-specific visual nuances by leveraging the subject and object's positional information and non-overlapping areas. Our method redefines entity pairs by incorporating the often-overlooked union region, excluding the intersection between the subject and object. Furthermore, we enhance the discriminative power among predicates by increasing the divergence between predicate-specific representations of the same entity pairs, thereby capturing the subtle visual nuances associated with each predicate. Extensive experiments on the Visual Genome dataset demonstrate that our approach achieves state-of-the-art performance.

关键词： scene graph generation Visual relation detection Predicate differentiation Non-overlapping regions Commonsense knowledge

来源：评论

学校读者我要写书评

暂无评论

Mining informativeness in scene graphs: Prioritizing informative relations in scene graph generation for enhanced performance in applications

引用

PATTERN RECOGNITION LETTERS 2025年 189卷 64-70页

作者： Neau, Maelic Santos, Paulo E. Bosser, Anne-Gwenn Macvicar, Alistair Buche, Cedric Flinders Univ S Australia Coll Sci & Engn Sturt Rd Adelaide SA 5042 Australia Ecole Natl Ingn Brest 945 Ave Technopole F-29280 Plouzane France Naval Grp Pacific Lot14 Adelaide SA 5000 Australia PrioriAnalytica 74 Pirie St Adelaide SA 5000 Australia CNRS UMR 6285 Lab STICC F-29280 Plouzane France

Learning to compose visual relationships from raw images in the form of scene graphs is a highly challenging Computer Vision task, yet it is essential for applications related to scene understanding. However, no current approaches in scene graph generation (SGG) aim at providing useful graphs for downstream tasks. Instead, the main focus has primarily been on unbiasing the data distribution for predicting more fine-grained relations. That being said, not all fine-grained relations are equally relevant to any particular task and at least a subset of them are of no use for real-world applications. In this work, we address the issue of the relevance of relations in scene graphs from the perspective of the quantity of information they bring to the understanding of the scene. To this end, we introduce anew evaluation metric for the task of SGG, called InformativeRecall@K, that aims at evaluating the ability of models to produce accurate and informative relations. We show that selecting relations based on this informativeness criteria is beneficial for the downstream tasks of Image generation, Visual Question Answering, and Image Captioning. Finally, we provide a new taxonomy of relations linked to the informativeness value for the task of Image generation.

关键词： scene graph generation scene Understanding Image Captioning Image generation Visual Question Answering

来源：评论

学校读者我要写书评

暂无评论

Semantic-enhanced panoptic scene graph generation through hybrid and axial attentions

引用

COMPLEX & INTELLIGENT SYSTEMS 2025年第1期11卷 1-15页

作者： Kuang, Xinhe Che, Yuxin Han, Huiyan Liu, Yimin Northeastern Univ Sydney Smart Technol Coll Qinhuangdao 066004 Peoples R China North Univ China Sch Comp Sci & Technol Taiyuan 030051 Peoples R China Shanxi Key Lab Machine Vis & Virtual Real Taiyuan 030051 Peoples R China Shanxi Ctr Technol Innovat Digital & Intelligent I Taiyuan 030031 Peoples R China Shanxi Cultural Tourism Grp Informat Technol Co Lt Taiyuan 030031 Peoples R China

The generation of panoramic scene graphs represents a cutting-edge challenge in image scene understanding, necessitating sophisticated predictions of both intra-object relationships and interactions between objects and their backgrounds. This complexity tests the limits of current predictive models' ability to discern nuanced relationships within images. Conventional approaches often fail to effectively combine visual and semantic data, leading to predictions that are semantically impoverished. To address these issues, we propose a novel method of semantic-enhanced panoramic scene graph generation through hybrid and axial attentions (PSGAtten). Specifically, a series of hybrid attention networks are stacked within both the object context encoding and relationship context encoding modules, enhancing the refinement and fusion of visual and semantic information. Within the hybrid attention networks, self-attention mechanisms facilitate feature refinement within modalities, while cross-attention mechanisms promote feature fusion across modalities. The axial attention model is further applied to enhance the integration ability of global information. Experimental validation on the PSG dataset confirms that our approach not only surpasses existing methods in generating detailed panoramic scene graphs but also significantly improves recall rates, thereby enhancing the ability to predict relationships in scene graph generation.

关键词： scene graph generation Panoptic scene graph Semantic information Attention mechanism

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：