arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.10312cs.CVeess.IV
收藏

OmniSSR:基于Stable Diffusion模型的零样本全向图像超分辨率

OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Model

School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院)

Runyi Li, Xuhan Sheng, Weiqi Li, Jian Zhang

AI总结:

针对现有全向图像超分辨率方法真实感差、泛化能力不足的问题,提出基于Stable Diffusion的零样本OmniSSR方法,通过投影转换、去噪迭代校正等实现高保真高真实感的超分辨率,无需训练且在基准数据集上验证有效。

摘要:

全向图像(ODIs)广泛应用于现实世界的视觉任务中,高分辨率全向图像有助于提升相关视觉任务的性能。现有大多数全向图像超分辨率方法采用端到端学习策略,导致生成图像的真实感较差,且训练方法缺乏有效的域外泛化能力。以扩散模型为代表的图像生成方法为视觉任务提供了强大的先验,且已被证明可有效应用于图像恢复任务。\n利用Stable Diffusion(SD,稳定扩散)模型的图像先验,我们实现了兼具保真度与真实感的全向图像超分辨率方法,命名为OmniSSR。首先,我们将等距柱状投影(ERP)图像转换为切平面投影(TP)图像,其分布近似于平面图像域。随后,我们使用SD迭代采样得到初始高分辨率结果。在每一步去噪迭代中,我们采用提出的十八重切向信息交互(OTII)与梯度分解(GD)技术对初始结果进行进一步校正和更新,以确保更好的一致性。最后,将TP图像转换回原投影形式,得到最终的高分辨率结果。\n我们的方法属于零样本方法,无需训练或微调。在两个基准数据集上开展的实验验证了所提方法的有效性。

英文摘要:

Omnidirectional images (ODIs) are commonly used in real-world visual tasks, and high-resolution ODIs help improve the performance of related visual tasks. Most existing super-resolution methods for ODIs use end-to-end learning strategies, resulting in inferior realness of generated images and a lack of effective out-of-domain generalization capabilities in training methods. Image generation methods represented by diffusion model provide strong priors for visual tasks and have been proven to be effectively applied to image restoration tasks. Leveraging the image priors of the Stable Diffusion (SD) model, we achieve omnidirectional image super-resolution with both fidelity and realness, dubbed as OmniSSR. Firstly, we transform the equirectangular projection (ERP) images into tangent projection (TP) images, whose distribution approximates the planar image domain. Then, we use SD to iteratively sample initial high-resolution results. At each denoising iteration, we further correct and update the initial results using the proposed Octadecaplex Tangent Information Interaction (OTII) and Gradient Decomposition (GD) technique to ensure better consistency. Finally, the TP images are transformed back to obtain the final high-resolution results. Our method is zero-shot, requiring no training or fine-tuning. Experiments of our method on two benchmark datasets demonstrate the effectiveness of our proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑