arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.11208cs.CV
收藏

THOR:通过关系干预的文本到人-物交互扩散模型

THOR: Text to Human-Object Interaction Diffusion via Relation Intervention

ShanghaiTech University(上海科技大学);Shanghai AI Laboratory(上海人工智能实验室)

Qianyang Wu, Ye Shi, Xiaoshui Huang, Jingyi Yu, Lan Xu, Jingya Wang

AI总结:

本文提出THOR扩散模型,通过关系干预机制增强人-物时空关系,并构建Text-BEHAVE数据集,有效解决文本生成动态人-物交互任务。

摘要:

本文提出了处理从文本描述生成动态人-物交互(Text2HOI)这一挑战性任务的新方法。现有的大多数工作假设交互仅涉及有限的身体部位或静态物体,而我们的任务需要同时应对人体运动的多样性、物体形状的多样性以及物体运动的语义模糊性。为了解决这一问题,我们提出了一种新颖的带有关系干预机制的文本引导人-物交互扩散模型(THOR)。THOR是一个配备关系干预机制的连贯扩散模型。在每个扩散步骤中,我们初始化文本引导的人体运动和物体运动,然后利用人-物关系对物体运动进行干预。这种干预增强了人与物体之间的时空关系,以人为中心的交互表示为从文本合成一致运动提供了额外指导。为了获得更合理和逼真的结果,我们在不同运动粒度级别引入了交互损失。此外,我们构建了Text-BEHAVE,一个Text2HOI数据集,该数据集将文本描述与当前最大的公开可用的3D HOI数据集无缝集成。定量和定性实验均证明了我们提出的模型的有效性。

英文摘要:

This paper addresses new methodologies to deal with the challenging task of generating dynamic Human-Object Interactions from textual descriptions (Text2HOI). While most existing works assume interactions with limited body parts or static objects, our task involves addressing the variation in human motion, the diversity of object shapes, and the semantic vagueness of object motion simultaneously. To tackle this, we propose a novel Text-guided Human-Object Interaction diffusion model with Relation Intervention (THOR). THOR is a cohesive diffusion model equipped with a relation intervention mechanism. In each diffusion step, we initiate text-guided human and object motion and then leverage human-object relations to intervene in object motion. This intervention enhances the spatial-temporal relations between humans and objects, with human-centric interaction representation providing additional guidance for synthesizing consistent motion from text. To achieve more reasonable and realistic results, interaction losses is introduced at different levels of motion granularity. Moreover, we construct Text-BEHAVE, a Text2HOI dataset that seamlessly integrates textual descriptions with the currently largest publicly available 3D HOI dataset. Both quantitative and qualitative experiments demonstrate the effectiveness of our proposed model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑