arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1612.00901cs.CVcs.AI
收藏

共同的不共同:情境识别中的语义稀疏性

Commonly Uncommon: Semantic Sparsity in Situation Recognition

University of Washington(华盛顿大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究院);University of Virginia(弗吉尼亚大学)

Mark Yatskar, Vicente Ordonez, Luke Zettlemoyer, Ali Farhadi

摘要:

语义稀疏性是结构化视觉分类问题中的一个常见挑战;当输出空间复杂时,绝大多数可能的预测很少出现在训练集中。本文研究了语义稀疏的情况下识别,产生结构化的总结图像中发生了什么,包括活动,对象和对象在活动中扮演的角色的任务。对于这个问题,我们发现经验,大多数对象-角色的组合是罕见的,目前最先进的模型显着表现不佳,在这种稀疏的数据制度。我们避免了许多这样的错误,通过(1)引入一个新的张量组合函数,学习在角色-名词组合之间共享示例,以及(2)使用Web数据自动收集很少观察到的输出示例,在语义上增强我们的训练数据。当集成在一个完整的基于CRF的结构化预测模型中时,基于张量的方法在前5名动词和名词角色准确度上分别比现有的最先进的方法提高了2.11%和4.40%。使用我们的语义增强技术添加500万张图像,在前5名动词和名词角色准确率上进一步相对提高了6.23%和9.57%。

英文摘要:

Semantic sparsity is a common challenge in structured visual classification problems; when the output space is complex, the vast majority of the possible predictions are rarely, if ever, seen in the training set. This paper studies semantic sparsity in situation recognition, the task of producing structured summaries of what is happening in images, including activities, objects and the roles objects play within the activity. For this problem, we find empirically that most object-role combinations are rare, and current state-of-the-art models significantly underperform in this sparse data regime. We avoid many such errors by (1) introducing a novel tensor composition function that learns to share examples across role-noun combinations and (2) semantically augmenting our training data with automatically gathered examples of rarely observed outputs using web data. When integrated within a complete CRF-based structured prediction model, the tensor-based approach outperforms existing state of the art by a relative improvement of 2.11% and 4.40% on top-5 verb and noun-role accuracy, respectively. Adding 5 million images with our semantic augmentation techniques gives further relative improvements of 6.23% and 9.57% on top-5 verb and noun-role accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑