文本控制的 Motion Mamba：文本指令的人体运动时序定位
Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion
AI总结：
本文提出基于文本的人体运动时序定位任务 THMG，构建线性内存成本的 TM-Mamba，并发布 BABEL-Grounding 数据集验证其有效性。
摘要：
人体运动理解是一项具有多种实际应用的基础任务，大规模动作捕捉数据集的可用性推动了其发展。近期研究聚焦于文本-运动任务，例如基于文本的运动生成、编辑和问答。在本研究中，我们提出了一项新颖的基于文本的人体运动定位（THMG）任务，旨在未裁剪的运动序列中精确定位与给定文本描述相对应的时间片段。捕获全局时间信息对 THMG 任务至关重要。然而，依赖全局时间自注意力的基于 Transformer 的模型在处理较长的未裁剪序列时，会因二次计算成本而面临挑战。为应对这些挑战，我们提出 Text-controlled Motion Mamba（TM-Mamba），这是一种统一模型，仅以线性内存成本整合时间全局上下文、语言查询控制和空间图拓扑。该模型的核心是文本控制选择机制，可根据文本查询动态融入全局时间信息。通过整合关系嵌入，模型进一步增强为具备拓扑感知能力。为进行评估，我们提出 BABEL-Grounding，这是首个提供人体动作详细文本描述及其对应时间片段的文本-运动数据集。大量评估证明了 TM-Mamba 在 BABEL-Grounding 上的有效性。
英文摘要：
Human motion understanding is a fundamental task with diverse practical applications, facilitated by the availability of large-scale motion capture datasets. Recent studies focus on text-motion tasks, such as text-based motion generation, editing and question answering. In this study, we introduce the novel task of text-based human motion grounding (THMG), aimed at precisely localizing temporal segments corresponding to given textual descriptions within untrimmed motion sequences. Capturing global temporal information is crucial for the THMG task. However, Transformer-based models that rely on global temporal self-attention face challenges when handling long untrimmed sequences due to the quadratic computational cost. We address these challenges by proposing Text-controlled Motion Mamba (TM-Mamba), a unified model that integrates temporal global context, language query control, and spatial graph topology with only linear memory cost. The core of the model is a text-controlled selection mechanism which dynamically incorporates global temporal information based on text query. The model is further enhanced to be topology-aware through the integration of relational embeddings. For evaluation, we introduce BABEL-Grounding, the first text-motion dataset that provides detailed textual descriptions of human actions along with their corresponding temporal segments. Extensive evaluations demonstrate the effectiveness of TM-Mamba on BABEL-Grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
