AIComposer：通过特征整合实现任意风格与内容的图像合成
AIComposer: Any Style and Content Image Composition via Feature Integration
AI总结：
本文提出无需文本提示的跨域图像合成方法 AIComposer，通过 MLP 整合前景与背景 CLIP 特征并结合局部交叉注意力，实现稳定风格化与无缝合成，同时发布评测基准并显著优于现有方法。
摘要：
借助大规模预训练文本到图像（T2I）扩散模型，图像合成已取得显著进展。尽管同域合成已有进步，但跨域合成仍未得到充分研究。主要挑战在于扩散模型的随机性以及输入图像之间的风格差距，这会导致失败和伪影。此外，对文本提示的严重依赖限制了实际应用。本文提出首个无需文本提示的跨域图像合成方法，可实现自然风格化与无缝合成。该方法高效且稳健，能够保留扩散先验，因为它只需少量反向反演和前向去噪步骤，且无需训练扩散器。该方法还使用简单的多层感知机网络整合来自前景和背景的 CLIP 特征，并通过局部交叉注意力策略操控扩散过程。它能有效保留前景内容，同时在没有预风格化网络的情况下实现稳定风格化。最后，我们构建了一个包含多样内容与风格的基准数据集以进行公平评估，解决跨域图像合成缺乏测试数据集的问题。在定性和定量评估中，我们的方法均优于最先进技术，使 LPIPS 分数显著提升 30.5%，CSD 指标提升 18.1%。我们相信该方法将推动未来研究与应用。代码和基准位于 https://github.com/sherlhw/AIComposer。
英文摘要：
Image composition has advanced significantly with large-scale pre-trained T2I diffusion models. Despite progress in same-domain composition, cross-domain composition remains under-explored. The main challenges are the stochastic nature of diffusion models and the style gap between input images, leading to failures and artifacts. Additionally, heavy reliance on text prompts limits practical applications. This paper presents the first cross-domain image composition method that does not require text prompts, allowing natural stylization and seamless compositions. Our method is efficient and robust, preserving the diffusion prior, as it involves minor steps for backward inversion and forward denoising without training the diffuser. Our method also uses a simple multilayer perceptron network to integrate CLIP features from foreground and background, manipulating diffusion with a local cross-attention strategy. It effectively preserves foreground content while enabling stable stylization without a pre-stylization network. Finally, we create a benchmark dataset with diverse contents and styles for fair evaluation, addressing the lack of testing datasets for cross-domain image composition. Our method outperforms state-of-the-art techniques in both qualitative and quantitative evaluations, significantly improving the LPIPS score by 30.5% and the CSD metric by 18.1%. We believe our method will advance future research and applications. Code and benchmark at https://github.com/sherlhw/AIComposer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
