MG-MotionLLM：一个跨多种粒度的动作理解与生成统一框架
MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularities
AI总结：
针对现有动作感知大语言模型仅关注粗粒度建模的局限，提出统一动作-语言模型 MG-MotionLLM，通过引入多粒度训练方案及辅助任务，实现跨粒度的动作理解与生成，在经典任务上表现优越并支持细粒度编辑。
摘要：
近期具备动作感知的大语言模型在统一动作理解与生成方面展现了可喜的潜力。然而，现有方法主要关注粗粒度的动作-文本建模，其中文本仅用寥寥数语描述整个动作序列的整体语义。这限制了它们处理细粒度动作相关任务的能力，例如理解和控制特定身体部位的运动。为了克服这一限制，我们率先提出了 MG-MotionLLM，一个用于多粒度动作理解和生成的统一动作-语言模型。我们进一步引入了一套全面的多粒度训练方案，通过整合一系列新颖的辅助任务，例如通过详细文本定位动作片段的时间边界以及动作详细描述，来促进跨不同粒度级别的动作-文本建模的相互增强。大量实验表明，我们的 MG-MotionLLM 在经典的文本到动作和动作到文本任务上取得了优越的性能，并在新颖的细粒度动作理解和编辑任务中展现出潜力。项目页面：CVI-SZU/MG-MotionLLM
英文摘要：
Recent motion-aware large language models have demonstrated promising potential in unifying motion comprehension and generation. However, existing approaches primarily focus on coarse-grained motion-text modeling, where text describes the overall semantics of an entire motion sequence in just a few words. This limits their ability to handle fine-grained motion-relevant tasks, such as understanding and controlling the movements of specific body parts. To overcome this limitation, we pioneer MG-MotionLLM, a unified motion-language model for multi-granular motion comprehension and generation. We further introduce a comprehensive multi-granularity training scheme by incorporating a set of novel auxiliary tasks, such as localizing temporal boundaries of motion segments via detailed text as well as motion detailed captioning, to facilitate mutual reinforcement for motion-text modeling across various levels of granularity. Extensive experiments show that our MG-MotionLLM achieves superior performance on classical text-to-motion and motion-to-text tasks, and exhibits potential in novel fine-grained motion comprehension and editing tasks. Project page: CVI-SZU/MG-MotionLLM
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
