GRAZE:基于 grounded 的细化与运动感知的零样本事件定位
GRAZE: Grounded Refinement and Motion-Aware Zero-Shot Event Localization
AI总结:
本文提出GRAZE,一种无需标注数据的零样本事件定位方法,通过结合Grounding DINO和SAM2实现运动感知的接触定位,有效应对复杂场景。
摘要:
美式足球训练生成大量视频,但感兴趣的互动仅占据每个长视频的短暂窗口。可靠的生物力学分析依赖于时空定位,以识别交互实体和接触开始。我们研究了First Point of Contact(FPOC),即球员首次触碰假人帧。在无约束训练视频中,我们提出了GRAZE,一种无需标注数据的FPOC定位管道。GRAZE使用Grounding DINO发现候选玩家-假人交互,通过运动感知的时间推理进行细化,并使用SAM2作为显式的像素级接触验证器。这种分离候选发现和接触确认的方法使方法在复杂场景和不稳定接地情况下更具鲁棒性。在738个打击练习视频上,GRAZE在97.4%的剪辑中产生有效输出,在77.5%的剪辑中将FPOC定位在±10帧内,在82.7%的剪辑中在±20帧内。这些结果表明,在真实世界训练视频中实现精确帧接触开始定位在没有任务特定训练的情况下是可行的。
英文摘要:
American football practice generates video at scale, yet the interaction of interest occupies only a brief window of each long, untrimmed clip. Reliable biomechanical analysis, therefore, depends on spatiotemporal localization that identifies both the interacting entities and the onset of contact. We study First Point of Contact (FPOC), defined as the first frame in which a player physically touches a tackle dummy, in unconstrained practice footage with camera motion, clutter, multiple similarly equipped athletes, and rapid pose changes around impact. We present GRAZE, a training-free pipeline for FPOC localization that requires no labeled tackle-contact examples. GRAZE uses Grounding DINO to discover candidate player-dummy interactions, refines them with motion-aware temporal reasoning, and uses SAM2 as an explicit pixel-level verifier of contact rather than relying on detection confidence alone. This separation between candidate discovery and contact confirmation makes the approach robust to cluttered scenes and unstable grounding near impact. On 738 tackle-practice videos, GRAZE produces valid outputs for 97.4% of clips and localizes FPOC within $\pm$ 10 frames on 77.5% of all clips and within $\pm$ 20 frames on 82.7% of all clips. These results show that frame-accurate contact onset localization in real-world practice footage is feasible without task-specific training.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。