全景图、3D场景和视频的统一编辑:通过解耦自注意力注入
Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection
AI总结:
提出一种仅用2D扩散模型的统一编辑框架,通过解耦自注意力注入实现全景图、3D场景和视频的语义一致编辑。
摘要:
虽然文本到图像模型在图像生成和编辑方面已展现出令人印象深刻的能力,但它们在各种模态上的应用通常需要训练单独的模型。受现有单图像编辑(通过自注意力注入)和视频编辑(通过共享注意力)方法的启发,我们提出了一种新颖的统一编辑框架,该框架结合了这两种方法的优势,仅利用一个基本的2D图像文本到图像(T2I)扩散模型。具体来说,我们设计了一种采样方法,该方法在参考图像和连续图像采样过程中利用共享的自注意力特征,从而在保持语义一致性的同时,促进连续图像的编辑。实验结果证实,我们的方法能够实现对包括3D场景、视频和全景图像在内的多种模态的编辑。
英文摘要:
While text-to-image models have achieved impressive capabilities in image generation and editing, their application across various modalities often necessitates training separate models. Inspired by existing method of single image editing with self attention injection and video editing with shared attention, we propose a novel unified editing framework that combines the strengths of both approaches by utilizing only a basic 2D image text-to-image (T2I) diffusion model. Specifically, we design a sampling method that facilitates editing consecutive images while maintaining semantic consistency utilizing shared self-attention features during both reference and consecutive image sampling processes. Experimental results confirm that our method enables editing across diverse modalities including 3D scenes, videos, and panorama images.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。