合并与分裂扩散路径以实现语义连贯的全景图
Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas
AI总结：
针对联合扩散生成全景图时语义不一致的问题，提出Merge-Attend-Diffuse算子，通过合并扩散路径并在聚合潜在空间重写注意力机制，有效提升了生成图像的语义连贯性且不牺牲视觉质量。
摘要：
扩散模型已成为文本到图像生成的最新技术，越来越多的研究致力于调整预训练扩散模型的推理过程以实现零样本能力。全景图生成是一个例子，近期工作通过在重叠的潜在特征上结合独立的扩散路径来解决此问题，这被称为联合扩散，能获得感知上对齐的全景图。然而，这些方法常产生语义不一致的输出，并以多样性换取一致性。为克服此局限，我们提出了Merge-Attend-Diffuse算子，可插入联合扩散设置中不同类型的预训练扩散模型，以提升生成全景图的感知与语义连贯性。具体而言，我们合并扩散路径，重写自注意力和交叉注意力以在聚合的潜在空间上运行。大量定量与定性实验分析及用户研究表明，该方法在保持与输入提示的兼容性和生成图像视觉质量的同时，提升了语义连贯性。代码发布于https://github.com/aimagelab/MAD。
英文摘要：
Diffusion models have become the State-of-the-Art for text-to-image generation, and increasing research effort has been dedicated to adapting the inference process of pretrained diffusion models to achieve zero-shot capabilities. An example is the generation of panorama images, which has been tackled in recent works by combining independent diffusion paths over overlapping latent features, which is referred to as joint diffusion, obtaining perceptually aligned panoramas. However, these methods often yield semantically incoherent outputs and trade-off diversity for uniformity. To overcome this limitation, we propose the Merge-Attend-Diffuse operator, which can be plugged into different types of pretrained diffusion models used in a joint diffusion setting to improve the perceptual and semantical coherence of the generated panorama images. Specifically, we merge the diffusion paths, reprogramming self- and cross-attention to operate on the aggregated latent space. Extensive quantitative and qualitative experimental analysis, together with a user study, demonstrate that our method maintains compatibility with the input prompt and visual quality of the generated images while increasing their semantic coherence. We release the code at https://github.com/aimagelab/MAD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
