Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation

作     者：Zheng, Chaofan Gao, Lianli Lyu, Xinyu Zeng, Pengpeng El Saddik, Abdulmotaleb Shen, Heng Tao 

作者机构：Future Media Center and School of Computer Science and Engineering University of Electronic Science and Technology of China Chengdu611731 China Multimedia Communications Research Laboratory and School of Electrical Engineering and Computer Science University of Ottawa OttawaONK1N 6N5 Canada 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2022年

核心收录：

主　　题：Learning systems 

摘      要：The current studies of Scene Graph Generation (SGG) focus on solving the long-tailed problem for generating unbiased scene graphs. However, most de-biasing methods overemphasize the tail predicates and underestimate head ones throughout training, thereby wrecking the representation ability of head predicate features. Furthermore, these impaired features from head predicates harm the learning of tail predicates. In fact, the inference of tail predicates heavily depends on the general patterns learned from head ones, e.g., standing on depends on on. Thus, these de-biasing SGG methods can neither achieve excellent performance on tail predicates nor satisfying behaviors on head ones. To address this issue, we propose a Dual-branch Hybrid Learning network (DHL) to take care of both head predicates and tail ones for SGG, including a Coarse-grained Learning Branch (CLB) and a Fine-grained Learning Branch (FLB). Specifically, the CLB is responsible for learning expertise and robust features of head predicates, while the FLB is expected to predict informative tail predicates. Furthermore, DHL is equipped with a Branch Curriculum Schedule (BCS) to make the two branches work well together. Experiments show that our approach achieves a new state-of-the-art performance on VG and GQA datasets and makes a trade-off between the performance of tail predicates and head ones. Moreover, extensive experiments on two downstream tasks (i.e., Image Captioning and Sentence-to-Graph Retrieval) further verify the generalization and practicability of our method. Our code is available at https://***/aa200647963/SGG-DHL/. Copyright © 2022, The Authors. All rights reserved.