arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.13083cs.CVcs.AI
收藏

人-物交互识别需要目标检测吗?

Is Object Detection Necessary for Human-Object Interaction Recognition?

Microsoft(微软);University of Washington(华盛顿大学)

Ying Jin, Yinpeng Chen, Lijuan Wang, Jianfeng Wang, Pei Yu, Zicheng Liu, Jenq-Neng Hwang

摘要:

本文重新审视人-物交互(HOI)识别在图像级不使用监督的对象的位置和人的姿态。我们称之为无检测的HOI识别,与现有的检测监督方法相比,这些方法依赖于对象和关键点检测来实现最先进的水平。使用我们的方法,不仅检测监督是不可避免的,而且通过适当地使用图像-文本预训练(如CLIP)和提出的对数和指数(LSE-Sign)损失函数可以实现优异的性能。具体来说,使用类标签的文本嵌入来初始化线性分类器对于利用CLIP预训练图像编码器至关重要。此外,LSE-Sign loss通过以softmax格式对所有类的梯度进行归一化,促进了从不平衡数据集上的多个标签中学习。令人惊讶的是,我们的免检测解决方案在CNOMO数据集上实现了60.5 mAP,比检测监督的最新技术水平高出13.4 mAP

英文摘要:

This paper revisits human-object interaction (HOI) recognition at image level without using supervisions of object location and human pose. We name it detection-free HOI recognition, in contrast to the existing detection-supervised approaches which rely on object and keypoint detections to achieve state of the art. With our method, not only the detection supervision is evitable, but superior performance can be achieved by properly using image-text pre-training (such as CLIP) and the proposed Log-Sum-Exp Sign (LSE-Sign) loss function. Specifically, using text embeddings of class labels to initialize the linear classifier is essential for leveraging the CLIP pre-trained image encoder. In addition, LSE-Sign loss facilitates learning from multiple labels on an imbalanced dataset by normalizing gradients over all classes in a softmax format. Surprisingly, our detection-free solution achieves 60.5 mAP on the HICO dataset, outperforming the detection-supervised state of the art by 13.4 mAP

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑