GIRAF:迈向与关节物体的可泛化人类交互
GIRAF: Towards Generalizable Human Interactions with Articulated Objects
AI总结:
针对合成与关节物体的逼真全身人类交互难题,现有模型有局限。本文提出文本条件扩散模型,通过以物体为中心的表示、混合域训练策略和基于接触的增强方案应对挑战,实验证明其对未见物体配置泛化能力强,超越现有方法。
摘要:
合成与关节物体逼真的全身人类交互是具身人工智能和图形学的一项基本挑战,应用于机器人训练和虚拟代理。现有模型存在局限,一些关注与静态物体的简单活动,另一些仅专注手部操作。本文提出生成协调全身运动的问题,关键在于联合推理运动、细粒度接触和物体关节。模型要捕捉跨物体几何形状的手部与物体对应关系,实现从导航到操作的无缝过渡。同时,大规模配对运动场景数据稀缺,难以泛化到不同物体位置和形状。我们引入文本条件扩散模型,通过以物体为中心的表示、混合域训练策略和基于接触的增强方案应对挑战。实验表明,该方法对未见物体配置具有强大的泛化能力,超越当前最先进方法。
英文摘要:
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation. This leaves open the problem of generating coordinated full-body motion that approaches, manipulates, and moves articulated objects in a realistic and generalizable way. The key difficulty lies in reasoning jointly about locomotion, fine-grained contact, and object articulation. Models must capture subtle hand-object correspondences that transfer across object geometries, while also producing seamless transitions from navigation to manipulation. At the same time, the scarcity of large-scale paired motion-scene data makes it difficult to generalize across diverse object positions and shapes. We introduce a text-conditioned diffusion model that addresses these challenges through three core ideas: an object-centric representation that unifies hand-object contact with object surfaces, a mixed-domain training strategy that balances locomotion and interaction, and a contact-based augmentation scheme that expands training diversity. Through experiments, our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。