CrossHOI-Bench: 一个统一的HOI评估基准，涵盖视觉-语言模型和特定HOI方法
CrossHOI-Bench: A Unified Benchmark for HOI Evaluation across Vision-Language Models and HOI-Specific Methods
AI总结：
本文提出CrossHOI-Bench，通过显式正例和 curated 负例，统一评估VLM和HOI方法，揭示两者在多任务和细粒度交互上的互补优劣。
摘要：
HOI检测长期由任务特定模型主导，有时使用早期视觉-语言backbone如CLIP。随着大生成VLMs的兴起，关键问题在于独立VLMs能否在专门的HOI方法面前竞争。现有基准如HICO-DET要求精确标签匹配，在不完整注释下任何不匹配的预测都被视为错误。这不公平地惩罚了有效输出，尤其是来自约束较少的VLMs，并使跨范式比较不可靠。为了解决这一限制，我们引入CrossHOI-Bench，一个多重选择的HOI基准，具有显式正例和精心挑选的负例，能够统一且可靠地评估VLMs和特定HOI模型。我们进一步聚焦于具有挑战性的场景，如多人物场景和细粒度交互区分，这对于揭示两种范式的实际差异至关重要。实验表明，大型VLMs在零样本性能上具有竞争力，有时甚至更优，但他们在多个并发动作和正确分配交互给目标人物方面挣扎。相反，特定HOI方法在一般HOI推理上仍较弱，但在多动作识别和更可靠地识别执行动作的人物方面表现更强。这些发现揭示了VLMs和特定HOI方法的互补优势和劣势，而现有基准由于错误的惩罚机制未能揭示。
英文摘要：
HOI detection has long been dominated by task-specific models, sometimes with early vision-language backbones such as CLIP. With the rise of large generative VLMs, a key question is whether standalone VLMs can perform HOI detection competitively against specialized HOI methods. Existing benchmarks such as HICO-DET require exact label matching under incomplete annotations, so any unmatched prediction is marked wrong. This unfairly penalizes valid outputs, especially from less constrained VLMs, and makes cross-paradigm comparison unreliable. To address this limitation, we introduce CrossHOI-Bench, a multiple-choice HOI benchmark with explicit positives and curated negatives, enabling unified and reliable evaluation of both VLMs and HOI-specific models. We further focus on challenging scenarios, such as multi-person scenes and fine-grained interaction distinctions, which are crucial for revealing real differences between the two paradigms. Experiments show that large VLMs achieve competitive, sometimes superior, zero-shot performance, yet they struggle with multiple concurrent actions and with correctly assigning interactions to the target person. Conversely, HOI-specific methods remain weaker in general HOI reasoning but demonstrate stronger multi-action recognition and more reliable identification of which person performs which action. These findings expose complementary strengths and weaknesses of VLMs and HOI-specific methods, which existing benchmarks fail to reveal due to incorrect penalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
