发表机构
AI Lab, CTO Division, LG Electronics（LG电子CTO部门AI实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多人场景中手部归属模糊问题，提出人物中心单查询框架，利用部件感知注意力与手-查询关系矩阵，统一实现双手交互检测、姿态估计与目标识别。
AI 中文摘要
理解人物级双手交互不仅需要检测手部，还需要识别哪两只手属于同一个人以及每只手与什么物体交互。现有的手-物交互方法大多是手中心的：它们将每只手视为独立实例，这可能导致多人场景中的归属模糊。我们提出了一种人物中心的公式化方法，其中单个查询预测一个人的结构化输出，包括人体框、身体姿态、手部框和状态以及交互目标。我们引入了部件感知的可变形注意力，以在人体、手部和姿态特定的参考区域之间分配注意力，使一个查询能够捕获完整的人物结构。我们进一步通过手到查询的关系矩阵统一了检测和交互推理，其中每只手从检测到的查询集加上一个可学习的关闭标记中选择其交互目标，直接恢复目标的框和类别，无需单独的物体回归。我们构建了一个基于COCO的数据集，包含人物中心的双手交互标注，并定义了用于评估手部状态和完整手-物元组的结构化指标。使用基于Transformer的检测器进行的实验表明，我们的公式化方法改进了人物级双手交互解析，并为联合检测、姿态估计和手部推理提供了有效的统一框架。
英文摘要
Understanding person-level bi-manual interactions requires not only detecting hands, but also identifying which two hands belong to the same person and what each hand interacts with. Existing hand--object interaction methods are mostly hand-centric: they treat each hand as an independent instance, which can lead to ambiguous ownership in multi-person scenes. We propose a person-centric formulation in which a single query predicts a structured output for one person, including the human box, body pose, hand boxes and states, and interaction targets. We introduce part-aware deformable attention to allocate attention across human, hand, and pose-specific reference regions, enabling one query to capture the full person structure. We further unify detection and interaction reasoning with a hand-to-query relationship matrix, where each hand selects its interaction target from the detected query set plus a learnable off token, directly recovering the target's box and class without separate object regression. We build a COCO-based dataset with person-centric bi-manual interaction annotations and define structured metrics for evaluating hand states and complete hand--object tuples. Experiments with a transformer-based detector show that our formulation improves person-level bi-manual interaction parsing and provides an effective unified framework for joint detection, pose estimation, and hand reasoning.
CommentsAccepted to ECCV2026, Project page: https://lgecto-ail-vil.github.io/SingleQuery-BHOI/
