Putting People in Their Place: 支持可供性感知的人物入景插入方法
Putting People in Their Place: Affordance-Aware Human Insertion into Scenes
AI总结：
该研究提出一种可供性感知的人物入景方法，通过自监督训练的大规模扩散模型实现符合场景约束的真实人物插入与交互合成，效果优于现有方法。
摘要：
我们通过提出一种将人物真实插入场景的方法，研究场景可供性推理问题。给定一张带有标记区域的场景图像和一张人物图像，我们在遵循场景可供性的前提下将人物插入场景。我们的模型能够根据场景上下文推理出一组真实姿态，调整参考人物的姿态，并对合成结果进行协调。我们通过学习视频片段中的人物重姿态，以自监督方式构建该任务。我们在包含240万个视频片段的数据集上训练了一个大规模扩散模型，该模型能在遵循场景上下文的同时生成多样且合理的姿态。基于学到的人物-场景合成能力，我们的模型还能在无约束条件下通过提示生成逼真的人物和场景，并支持交互式编辑。定量评估表明，与现有工作相比，我们的方法合成的人物外观更真实，人物-场景交互更自然。
英文摘要：
We study the problem of inferring scene affordances by presenting a method for realistically inserting people into scenes. Given a scene image with a marked region and an image of a person, we insert the person into the scene while respecting the scene affordances. Our model can infer the set of realistic poses given the scene context, re-pose the reference person, and harmonize the composition. We set up the task in a self-supervised fashion by learning to re-pose humans in video clips. We train a large-scale diffusion model on a dataset of 2.4M video clips that produces diverse plausible poses while respecting the scene context. Given the learned human-scene composition, our model can also hallucinate realistic people and scenes when prompted without conditioning and also enables interactive editing. A quantitative evaluation shows that our method synthesizes more realistic human appearance and more natural human-scene interactions than prior work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
