开放词汇人-物交互检测：交互感知提示与概念校准
Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibration
AI总结：
提出INP-CC，一种端到端开放词汇HOI检测器，通过交互感知提示生成器和语言模型引导的概念校准，在SWIG-HOI和HICO-DET上超越现有方法。
摘要：
开放词汇人-物交互（HOI）检测旨在检测人与物体之间的交互，同时泛化到训练集之外的新颖交互类别。当前方法通常依赖视觉与语言模型（VLMs），但由于图像编码器次优而面临挑战，因为图像级预训练与HOI所需的细粒度区域级交互检测对齐不佳。此外，有效编码视觉外观的文本描述仍然困难，限制了模型捕获详细HOI关系的能力。为解决这些问题，我们提出了交互感知提示与概念校准（INP-CC），一种端到端的开放词汇HOI检测器，它整合了交互感知提示和概念校准。具体而言，我们提出了一种交互感知提示生成器，根据输入场景动态生成一组紧凑的提示，使得相似交互之间能够选择性共享。这种方法将模型的注意力引导到关键交互模式而非通用图像级语义上，从而增强HOI检测。此外，我们通过语言模型引导的校准来细化HOI概念表示，通过调查类别间的视觉相似性来帮助区分多样的HOI概念。还采用了负采样策略来改进模态间相似性建模，使模型能够更好地区分视觉上相似但语义上不同的动作。大量实验结果表明，INP-CC在SWIG-HOI和HICO-DET数据集上显著优于最先进的模型。代码可在https://github.com/ltttpku/INP-CC获取。
英文摘要：
Open Vocabulary Human-Object Interaction (HOI) detection aims to detect interactions between humans and objects while generalizing to novel interaction classes beyond the training set. Current methods often rely on Vision and Language Models (VLMs) but face challenges due to suboptimal image encoders, as image-level pre-training does not align well with the fine-grained region-level interaction detection required for HOI. Additionally, effectively encoding textual descriptions of visual appearances remains difficult, limiting the model's ability to capture detailed HOI relationships. To address these issues, we propose INteraction-aware Prompting with Concept Calibration (INP-CC), an end-to-end open-vocabulary HOI detector that integrates interaction-aware prompts and concept calibration. Specifically, we propose an interaction-aware prompt generator that dynamically generates a compact set of prompts based on the input scene, enabling selective sharing among similar interactions. This approach directs the model's attention to key interaction patterns rather than generic image-level semantics, enhancing HOI detection. Furthermore, we refine HOI concept representations through language model-guided calibration, which helps distinguish diverse HOI concepts by investigating visual similarities across categories. A negative sampling strategy is also employed to improve inter-modal similarity modeling, enabling the model to better differentiate visually similar but semantically distinct actions. Extensive experimental results demonstrate that INP-CC significantly outperforms state-of-the-art models on the SWIG-HOI and HICO-DET datasets. Code is available at https://github.com/ltttpku/INP-CC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
