ACD:基于图像-句子语料库的动作概念发现
ACD: Action Concept Discovery from Image-Sentence Corpora
摘要:
静止图像中的动作分类是计算机视觉中的一项重要任务。这是具有挑战性的,因为动作的外观可以根据它们的上下文(例如,相关联的对象)而变化。上下文信息的手动标记将是耗时的并且难以扩展。为了解决这一挑战,我们提出了一种方法来自动发现和聚类动作概念,并从弱监督的图像句子语料库中学习它们的分类器。它通过从句子中提取动宾对来获得候选动作概念,并使用关联图像来验证其可视性。然后,通过使用多模态表示与来自深度卷积网络的图像嵌入和来自word 2 vec的文本嵌入来聚类候选动作概念。一百多人的动作概念分类器从Flickr 30 k数据集学习,没有额外的人力和有前途的分类结果。我们进一步应用AdaBoost算法自动选择和组合相关的动作概念给出一个动作查询。在PASCAL VOC 2012动作分类基准上显示了有希望的结果,该基准与Flickr 30 k没有重叠。
英文摘要:
Action classification in still images is an important task in computer vision. It is challenging as the appearances of ac- tions may vary depending on their context (e.g. associated objects). Manually labeling of context information would be time consuming and difficult to scale up. To address this challenge, we propose a method to automatically discover and cluster action concepts, and learn their classifiers from weakly supervised image-sentence corpora. It obtains candidate action concepts by extracting verb-object pairs from sentences and verifies their visualness with the associated images. Candidate action concepts are then clustered by using a multi-modal representation with image embeddings from deep convolutional networks and text embeddings from word2vec. More than one hundred human action concept classifiers are learned from the Flickr 30k dataset with no additional human effort and promising classification results are obtained. We further apply the AdaBoost algorithm to automatically select and combine relevant action concepts given an action query. Promising results have been shown on the PASCAL VOC 2012 action classification benchmark, which has zero overlap with Flickr30k.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。