arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.09786cs.CVcs.AI
收藏

面向细粒度场景图生成的自适应自训练框架

Adaptive Self-training Framework for Fine-grained Scene Graph Generation

KAIST(韩国科学技术院);ETRI(韩国电子通信研究院);Korea University(高丽大学)

Kibum Kim, Kanghoon Yoon, Yeonjun In, Jinyoung Moon, Donghyun Kim, Chanyoung Park

AI总结:

本文提出ST-SGG自训练框架,通过CATM伪标签技术和图结构学习器缓解场景图生成中的长尾分布与标注缺失问题,有效提升细粒度谓词类别的性能。

摘要:

场景图生成(SGG)模型一直受到基准数据集固有问题的困扰,例如谓词长尾分布和标注缺失问题。在本工作中,我们旨在通过利用未标注三元组来缓解SGG的长尾问题。为此,我们引入了一种用于SGG的自训练框架(ST-SGG),该框架为未标注三元组分配伪标签,并基于这些伪标签训练SGG模型。尽管自训练在图像识别领域已取得显著进展,但由于SGG任务的内在特性(如语义歧义和谓词类别的长尾分布),为其设计自训练框架更具挑战性。因此,我们提出了一种新颖的SGG伪标签技术,称为基于动量的类别自适应阈值法(CATM),这是一个与模型无关的框架,可应用于任何现有的SGG模型。此外,我们设计了一个图结构学习器(GSL),当将我们提出的自训练框架应用于最先进的基于消息传递神经网络(MPNN)的SGG模型时,该学习器十分有益。我们的大量实验验证了ST-SGG在各种SGG模型上的有效性,特别是在提升细粒度谓词类别的性能方面。

英文摘要:

Scene graph generation (SGG) models have suffered from inherent problems regarding the benchmark datasets such as the long-tailed predicate distribution and missing annotation problems. In this work, we aim to alleviate the long-tailed problem of SGG by utilizing unannotated triplets. To this end, we introduce a Self-Training framework for SGG (ST-SGG) that assigns pseudo-labels for unannotated triplets based on which the SGG models are trained. While there has been significant progress in self-training for image recognition, designing a self-training framework for the SGG task is more challenging due to its inherent nature such as the semantic ambiguity and the long-tailed distribution of predicate classes. Hence, we propose a novel pseudo-labeling technique for SGG, called Class-specific Adaptive Thresholding with Momentum (CATM), which is a model-agnostic framework that can be applied to any existing SGG models. Furthermore, we devise a graph structure learner (GSL) that is beneficial when adopting our proposed self-training framework to the state-of-the-art message-passing neural network (MPNN)-based SGG models. Our extensive experiments verify the effectiveness of ST-SGG on various SGG models, particularly in enhancing the performance on fine-grained predicate classes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑