零样本开放词汇人体运动接地与测试时训练
Zero-Shot Open-Vocabulary Human Motion Grounding with Test-Time Training
AI总结：
本文提出ZOMG框架，通过语言语义分区和软掩码优化，在无需标注或微调的情况下实现零样本开放词汇的人体运动接地，提升了运动理解的性能和效率。
摘要：
理解复杂的人体活动需要能够将运动分解为细粒度、语义对齐的子动作的能力。这一运动接地过程对于行为分析、具身人工智能和虚拟现实至关重要。然而，大多数现有方法依赖于密集监督和预定义的动作类别，这在开放词汇、现实世界环境中不可行。在本文中，我们提出了ZOMG，一个零样本、开放词汇框架，能够在不需任何标注或微调的情况下将运动序列分割成具有语义意义的子动作。技术上，ZOMG整合了（1）语言语义分区，利用大型语言模型将指令分解为有序的子动作单元，以及（2）软掩码优化，学习实例特定的时间掩码以聚焦于对子动作关键的帧，同时保持段内连续性和段间分离性，所有这些都不改变预训练编码器。在三个运动-语言数据集上的实验表明，运动接地性能的最先进效果和效率，优于先前方法在HumanML3D基准测试中提升了+8.7%的mAP。同时，在下游检索中也取得了显著改进，建立了无标注运动理解的新范式。
英文摘要：
Understanding complex human activities demands the ability to decompose motion into fine-grained, semantic-aligned sub-actions. This motion grounding process is crucial for behavior analysis, embodied AI and virtual reality. Yet, most existing methods rely on dense supervision with predefined action classes, which are infeasible in open-vocabulary, real-world settings. In this paper, we propose ZOMG, a zero-shot, open-vocabulary framework that segments motion sequences into semantically meaningful sub-actions without requiring any annotations or fine-tuning. Technically, ZOMG integrates (1) language semantic partition, which leverages large language models to decompose instructions into ordered sub-action units, and (2) soft masking optimization, which learns instance-specific temporal masks to focus on frames critical to sub-actions, while maintaining intra-segment continuity and enforcing inter-segment separation, all without altering the pretrained encoder. Experiments on three motion-language datasets demonstrate state-of-the-art effectiveness and efficiency of motion grounding performance, outperforming prior methods by +8.7\% mAP on HumanML3D benchmark. Meanwhile, significant improvements also exist in downstream retrieval, establishing a new paradigm for annotation-free motion understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
