Refaçade: 基于给定参考纹理的物体编辑
Refaçade: Editing Object with Given Reference Texture
AI总结：
Refaçade通过纹理去除和局部纹理统计增强，实现精确可控的物体纹理迁移。
摘要：
最近扩散模型的进步在图像和视频编辑领域带来了显著进展，但某些任务仍处于未充分探索的状态。在本文中，我们引入了一个新任务，即物体纹理迁移，其目标是将参考物体的局部纹理转移到图像或视频中的目标物体上。为了完成这一任务，一种直接的解决方案是使用受源结构和参考纹理条件的ControlNet。然而，这种方法有两个缺点：对原始参考图像进行条件化会引入不需要的结构信息，并且无法分离源物体的视觉纹理和结构信息。为了解决这个问题，我们提出了Refaçade，该方法包含两个关键设计，以实现图像和视频中精确且可控的纹理迁移。首先，我们使用一个在配对的有纹理/无纹理3D网格渲染上训练的纹理去除器，以去除外观信息，同时保持源视频的几何和运动。其次，我们通过使用图片拼图排列来破坏参考的全局布局，促使模型关注局部纹理统计，而不是物体的全局布局。广泛的实验表明，该方法在视觉质量和精确编辑方面具有优越性，并且具有可控性，在定量和人类评估中均优于强大的基线方法。代码可在https://github.com/fishZe233/Refacade上获得。
英文摘要：
Recent advances in diffusion models have brought remarkable progress in image and video editing, yet some tasks remain underexplored. In this paper, we introduce a new task, Object Retexture, which transfers local textures from a reference object to a target object in images or videos. To perform this task, a straightforward solution is to use ControlNet conditioned on the source structure and the reference texture. However, this approach suffers from limited controllability for two reasons: conditioning on the raw reference image introduces unwanted structural information, and it fails to disentangle the visual texture and structure information of the source. To address this problem, we propose Refaçade, a method that consists of two key designs to achieve precise and controllable texture transfer in both images and videos. First, we employ a texture remover trained on paired textured/untextured 3D mesh renderings to remove appearance information while preserving the geometry and motion of source videos. Second, we disrupt the reference global layout using a jigsaw permutation, encouraging the model to focus on local texture statistics rather than the global layout of the object. Extensive experiments demonstrate superior visual quality, precise editing, and controllability, outperforming strong baselines in both quantitative and human evaluations. Code is available at https://github.com/fishZe233/Refacade.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
