DiT360：通过混合训练实现高保真全景图像生成
DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training
AI总结：
本文提出DiT360框架，通过在透视和全景数据上混合训练，结合跨域知识注入与多层监督策略，解决了全景图像生成中的几何保真度与照片级真实感问题。
摘要：
在本工作中，我们提出了DiT360，这是一个基于DiT的框架，通过对透视数据和全景数据进行混合训练来生成全景图像。针对生成质量中保持几何保真度和照片级真实感的问题，我们将主要原因归结为缺乏大规模、高质量、真实的现实世界全景数据，其中这种以数据为中心的方法与以往专注于模型设计的方法不同。基本上，DiT360在预VAE图像层和后VAE token层都应用了几个用于域间变换和域内增强的关键模块。在图像层，我们通过透视引导和全景细化来注入跨域知识，在增强感知内容的同时调节分布以实现照片级真实感。在token层，混合监督被应用于多个模块，包括用于边界连续性的圆形填充、用于旋转鲁棒性的偏航损失，以及用于失真感知的立方体损失。在文本到全景的修复和外绘任务上的大量实验表明，我们的方法在十一项定量指标上实现了更好的边界一致性和图像保真度。我们的代码可在https://github.com/Insta360-Research-Team/diT360获取。
英文摘要：
In this work, we propose DiT360, a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation. For the issues of maintaining geometric fidelity and photorealism in generation quality, we attribute the main reason to the lack of large-scale, high-quality, real-world panoramic data, where such a data-centric view differs from prior methods that focus on model design. Basically, DiT360 has several key modules for inter-domain transformation and intra-domain augmentation, applied at both the pre-VAE image level and the post-VAE token level. At the image level, we incorporate cross-domain knowledge through perspective image guidance and panoramic refinement, which enhance perceptual quality while regularizing diversity and photorealism. At the token level, hybrid supervision is applied across multiple modules, which include circular padding for boundary continuity, yaw loss for rotational robustness, and cube loss for distortion awareness. Extensive experiments on text-to-panorama, inpainting, and outpainting tasks demonstrate that our method achieves better boundary consistency and image fidelity across eleven quantitative metrics. Our code is available at https://github.com/Insta360-Research-Team/DiT360.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
