arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2011.10084cs.CVcs.LG
收藏

基于注意力的场景图分类

Classification by Attention: Scene Graph Classification with Prior Knowledge

Ludwig Maximilian University of Munich(慕尼黑大学);Sapienza University of Rome(罗马大学);Siemens AG(西门子公司)

Sahand Sharifzadeh, Sina Moayed Baharlou, Volker Tresp

摘要:

场景图分类中的一个主要挑战是,对象和关系的外观可以从一个图像到另一个显着不同。以前的作品已经解决了这个问题的关系推理在图像中的所有对象或将先验知识分类。与以前的作品不同,我们不考虑感知和先验知识的单独模型。相反,我们采用多任务学习方法,将分类实现为注意力层。这允许先验知识在感知模型中出现和传播。通过强制模型也代表先验,我们实现了强归纳偏差。我们表明,我们的模型可以准确地生成常识知识,这种知识的场景表示的迭代注入导致显着更高的分类性能。此外,我们的模型可以根据三元组给出的外部知识进行微调。当与自监督学习相结合时,仅使用1%的注释图像,这使得对象分类提高了3%以上,场景图分类提高了26%,谓词预测准确率提高了36%。

英文摘要:

A major challenge in scene graph classification is that the appearance of objects and relations can be significantly different from one image to another. Previous works have addressed this by relational reasoning over all objects in an image or incorporating prior knowledge into classification. Unlike previous works, we do not consider separate models for perception and prior knowledge. Instead, we take a multi-task learning approach, where we implement the classification as an attention layer. This allows for the prior knowledge to emerge and propagate within the perception model. By enforcing the model also to represent the prior, we achieve a strong inductive bias. We show that our model can accurately generate commonsense knowledge and that the iterative injection of this knowledge to scene representations leads to significantly higher classification performance. Additionally, our model can be fine-tuned on external knowledge given as triples. When combined with self-supervised learning and with 1% of annotated images only, this gives more than 3% improvement in object classification, 26% in scene graph classification, and 36% in predicate prediction accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑