发表机构
KTH Royal Institute of Technology; MPI for Informatics; University of Cambridge（瑞典皇家理工学院; 马克斯·普朗克信息学研究所; 剑桥大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对空间锚定手势生成，提出含约2000个指向标注片段的基准及评估协议，并给出流匹配基线，发现几何锚定可超越人类而自然度不变。
AI 中文摘要
在共享空间中的交流交织着语言和非语言信号，而指向手势将语言锚定于环境：“把杯子放在那个上面”如果没有固定所指对象的手势，就无法解释。然而，目前尚无通用框架来评估生成的手势是否指示其预期的所指对象；分布性指标会奖励指向错误对象的手势，只要它看起来自然即可。我们引入了一个用于空间锚定手势生成的基准，包含来自自然对话的约2000个带指向标注的片段，并带有真实的三维所指对象；任务要求系统在对话语音中决定何时、如何以及在哪里指向；协议则分别评估时间对齐、空间锚定和感知自然度。我们还提供了一个流匹配基线，即MM-Conv-Flow。将其与一个独立的基于检索的系统以及捕捉到的人类运动进行比较，我们发现几何锚定可以超过人类指向的水平，而感知自然度并无提升，这表明参照性手势的质量必须沿多个维度进行衡量。
英文摘要
Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: "put the cup on that one" is uninterpretable without the gesture that fixes the referent. Yet no common framework exists for evaluating whether generated gestures indicate their intended referent; distributional metrics reward a gesture aimed at the wrong object as long as it looks natural. We introduce a benchmark for spatially grounded gesture generation, comprising ~2K pointing-annotated clips from naturalistic VR dialogue with ground-truth 3D referents, a task in which systems must decide when, how and where to point within conversational speech, and a protocol that separates temporal alignment, spatial grounding and perceived naturalness. We also provide a flow-matching baseline, MM-Conv-Flow. Evaluating it alongside an independent retrieval-based system and captured human motion, we find that geometric grounding can exceed that of human pointing without any gain in perceived naturalness, showing that referential gesture quality must be measured along separate dimensions.
Comments13 pages, 9 figures. Benchmark of the Referential Gesture Challenge at the HSI Workshop, ECCV 2026. Data and video: https://huggingface.co/datasets/hsi-workshop/referential-gesture-challenge
