MVDiffusion：利用对应感知扩散实现整体多视图图像生成
MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion
AI总结：
MVDiffusion提出一种对应感知的扩散方法，通过并行生成和跨视图注意力，实现从文本提示生成一致的多视图图像，解决误差累积问题，并在全景图及深度到图像生成任务中达到先进水平。
摘要：
本文介绍了MVDiffusion，一种简单而有效的方法，用于在给定像素到像素对应关系（例如，来自全景图的透视裁剪或给定深度图和姿态的多视图图像）的情况下，从文本提示生成一致的多视图图像。与依赖迭代图像扭曲和修复的先前方法不同，MVDiffusion以全局感知同时生成所有图像，有效解决了普遍存在的误差累积问题。其核心是，MVDiffusion使用预训练的文本到图像扩散模型并行处理透视图像，同时集成新颖的对应感知注意力层以促进跨视图交互。对于全景图生成，尽管仅使用10k个全景图进行训练，MVDiffusion仍能为任意文本生成高分辨率逼真图像，或将一个透视图像外推到360度视图。对于多视图深度到图像生成，MVDiffusion在纹理化场景网格方面展示了最先进的性能。
英文摘要：
This paper introduces MVDiffusion, a simple yet effective method for generating consistent multi-view images from text prompts given pixel-to-pixel correspondences (e.g., perspective crops from a panorama or multi-view images given depth maps and poses). Unlike prior methods that rely on iterative image warping and inpainting, MVDiffusion simultaneously generates all images with a global awareness, effectively addressing the prevalent error accumulation issue. At its core, MVDiffusion processes perspective images in parallel with a pre-trained text-to-image diffusion model, while integrating novel correspondence-aware attention layers to facilitate cross-view interactions. For panorama generation, while only trained with 10k panoramas, MVDiffusion is able to generate high-resolution photorealistic images for arbitrary texts or extrapolate one perspective image to a 360-degree view. For multi-view depth-to-image generation, MVDiffusion demonstrates state-of-the-art performance for texturing a scene mesh.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
