You Think, You ACT:任意文本到动作生成的新任务
You Think, You ACT: The New Task of Arbitrary Text to Motion Generation
AI总结:
本文将文本到动作任务扩展至任意文本,构建了HUMANML3D++数据集,提出从任意文本提取动作指令并生成动作的框架,并引入多解指标进行基准测试,促进该实用方向的新研究。
摘要:
文本到动作旨在从文本生成人体动作。现有设置依赖于包含动作标签的有限动作文本,这限制了在难以直接描述的场景中的灵活性和实用性。本文将有限的动作文本扩展为任意文本。没有显式动作标签的场景文本能够增强模型在虚拟人交互、机器人行为生成和电影制作等复杂多样行业中的实用性,同时也支持对潜在的隐式行为模式的探索。然而,新引入的场景文本可能产生多个合理的输出结果,给现有数据、框架和评估带来重大挑战。为解决这一实际问题,我们首先通过扩展现有最大数据集HUMANML3D的文本,创建了新数据集HUMANML3D++。其次,我们提出了一个简单而有效的框架,从任意文本中提取动作指令,随后生成动作。此外,我们还使用多解指标对该新设置进行基准测试,以解决现有单解指标的不足。大量实验表明,在这种现实设置下的文本到动作任务具有挑战性,促进了这一实用方向的新研究。
英文摘要:
Text to Motion aims to generate human motions from texts. Existing settings rely on limited Action Texts that include action labels, which limits flexibility and practicability in scenarios difficult to describe directly. This paper extends limited Action Texts to arbitrary ones. Scene texts without explicit action labels can enhance the practicality of models in complex and diverse industries such as virtual human interaction, robot behavior generation, and film production, while also supporting the exploration of potential implicit behavior patterns. However, newly introduced Scene Texts may yield multiple reasonable output results, causing significant challenges in existing data, framework, and evaluation. To address this practical issue, we first create a new dataset HUMANML3D++ by extending texts of the largest existing dataset HUMANML3D. Secondly, we propose a simple yet effective framework that extracts action instructions from arbitrary texts and subsequently generates motions. Furthermore, we also benchmark this new setting with multi-solution metrics to address the inadequacies of existing single-solution metrics. Extensive experiments indicate that Text to Motion in this realistic setting is challenging, fostering new research in this practical direction.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。