arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.08397cs.CVcs.AI
收藏

360PanT:无需训练的文本驱动360度全景图到全景图翻译

360PanT: Training-Free Text-Driven 360-Degree Panorama-to-Panorama Translation

University College London(伦敦大学学院)

Hai Wang, Jing-Hao Xue

AI总结:

针对文本驱动360度全景图翻译存在的边界不连续问题,提出无需训练的360PanT方法,通过边界连续性编码与带空间控制的无缝平铺翻译实现无缝效果,经多数据集验证有效并已开源代码。

摘要:

在360度全景图的翻译过程中保持边界连续性,对现有文本驱动的图像到图像翻译方法而言仍是一项重大挑战。这类方法往往会在翻译后的全景图边界处产生视觉上突兀的不连续效果,破坏沉浸体验。为解决这一问题,我们提出360PanT,这是一种无需训练的、基于文本的360度全景图到全景图翻译方法,可实现边界连续性。我们的360PanT通过两个核心组件实现无缝翻译:边界连续性编码,以及带空间控制的无缝平铺翻译。首先,边界连续性编码通过构建扩展输入图像,将输入360度全景图的关键边界连续性信息嵌入到含噪潜表示中。其次,借助该嵌入的含噪潜表示,并在目标提示的引导下,带空间控制的无缝平铺翻译能够生成左右两半完全一致的翻译图像,同时遵循扩展输入的结构与语义布局。这一过程确保最终翻译出的360度全景图具备无缝的边界连续性。在真实世界数据集与合成数据集上的实验结果,证明了我们的360PanT在360度全景图翻译方面的有效性。代码可在https://github.com/littlewhitesea/360PanT 获取。

英文摘要:

Preserving boundary continuity in the translation of 360-degree panoramas remains a significant challenge for existing text-driven image-to-image translation methods. These methods often produce visually jarring discontinuities at the translated panorama's boundaries, disrupting the immersive experience. To address this issue, we propose 360PanT, a training-free approach to text-based 360-degree panorama-to-panorama translation with boundary continuity. Our 360PanT achieves seamless translations through two key components: boundary continuity encoding and seamless tiling translation with spatial control. Firstly, the boundary continuity encoding embeds critical boundary continuity information of the input 360-degree panorama into the noisy latent representation by constructing an extended input image. Secondly, leveraging this embedded noisy latent representation and guided by a target prompt, the seamless tiling translation with spatial control enables the generation of a translated image with identical left and right halves while adhering to the extended input's structure and semantic layout. This process ensures a final translated 360-degree panorama with seamless boundary continuity. Experimental results on both real-world and synthesized datasets demonstrate the effectiveness of our 360PanT in translating 360-degree panoramas. Code is available at \href{https://github.com/littlewhitesea/360PanT}{https://github.com/littlewhitesea/360PanT}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑