面向统一多模态模型的通用多图像编辑
Towards Generalized Multi-Image Editing for Unified Multimodal Models
AI总结：
本文提出了一种面向统一多模态模型的通用多图像编辑框架，通过可学习的潜在分离器和正弦索引编码提升多图像编辑任务中的视觉一致性和泛化能力。
摘要：
统一多模态模型（UMMs）集成了多模态理解和生成，但它们在跨多个输入图像引用细节时，难以保持视觉一致性和消歧视觉线索。在本文中，我们提出了一种可扩展的多图像编辑框架，用于UMMs，该框架明确区分图像身份并能适应可变的输入数量。算法上，我们引入了两项创新：1）可学习的潜在分离器明确区分每个参考图像在潜在空间中的特征，从而实现准确且解耦的条件设置。2）正弦索引编码为同一图像的视觉令牌分配连续的正弦索引嵌入，这提供了显式的图像身份，同时允许在可变输入数量上进行泛化和外推。为了便于训练和评估，我们建立了一个高保真度的基准，使用逆向数据集构建方法来确保无瑕疵、可实现的输出。实验表明，在多样化的多图像编辑任务上，与先前基线相比，语义一致性、视觉保真度和跨图像整合方面有明显提升，验证了我们在一致性和泛化能力上的优势。
英文摘要：
Unified Multimodal Models (UMMs) integrate multimodal understanding and generation, yet they are limited to maintaining visual consistency and disambiguating visual cues when referencing details across multiple input images. In this work, we propose a scalable multi-image editing framework for UMMs that explicitly distinguishes image identities and generalizes to variable input counts. Algorithmically, we introduce two innovations: 1) The learnable latent separators explicitly differentiate each reference image in the latent space, enabling accurate and disentangled conditioning. 2) The sinusoidal index encoding assigns visual tokens from the same image a continuous sinusoidal index embedding, which provides explicit image identity while allowing generalization and extrapolation on a variable number of inputs. To facilitate training and evaluation, we establish a high-fidelity benchmark using an inverse dataset construction methodology to guarantee artifact-free, achievable outputs. Experiments show clear improvements in semantic consistency, visual fidelity, and cross-image integration over prior baselines on diverse multi-image editing tasks, validating our advantages on consistency and generalization ability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
