Mining informativeness in scene graphs: Prioritizing informative relations in Scene Graph Generation for enhanced performance in applications

作     者：Neau, Maelic Santos, Paulo E. Bosser, Anne-Gwenn Macvicar, Alistair Buche, Cedric 

作者机构：Flinders Univ S Australia Coll Sci & Engn Sturt Rd Adelaide SA 5042 Australia Ecole Natl Ingn Brest 945 Ave Technopole F-29280 Plouzane France Naval Grp Pacific Lot14 Adelaide SA 5000 Australia PrioriAnalytica 74 Pirie St Adelaide SA 5000 Australia CNRS UMR 6285 Lab STICC F-29280 Plouzane France 

出 版 物：《PATTERN RECOGNITION LETTERS》 (Pattern Recogn. Lett.)

年 卷 期：2025年第189卷

页      面：64-70页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：Scene Graph Generation Scene Understanding Image Captioning Image Generation Visual Question Answering 

摘      要：Learning to compose visual relationships from raw images in the form of scene graphs is a highly challenging Computer Vision task, yet it is essential for applications related to scene understanding. However, no current approaches in Scene Graph Generation (SGG) aim at providing useful graphs for downstream tasks. Instead, the main focus has primarily been on unbiasing the data distribution for predicting more fine-grained relations. That being said, not all fine-grained relations are equally relevant to any particular task and at least a subset of them are of no use for real-world applications. In this work, we address the issue of the relevance of relations in Scene Graphs from the perspective of the quantity of information they bring to the understanding of the scene. To this end, we introduce anew evaluation metric for the task of SGG, called InformativeRecall@K, that aims at evaluating the ability of models to produce accurate and informative relations. We show that selecting relations based on this informativeness criteria is beneficial for the downstream tasks of Image Generation, Visual Question Answering, and Image Captioning. Finally, we provide a new taxonomy of relations linked to the informativeness value for the task of Image Generation.