OmniSplat：驯服前馈3D高斯泼溅以实现具有可编辑能力的全向图像
OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities
- Seoul National University（首尔大学）
- LG AI Research（LG AI研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对全向图像，提出无需训练的前馈3DGS框架OmniSplat，采用阴阳网格分解图像以减少域差距，提升重建精度并支持可编辑能力。
AI中文摘要：
前馈3D高斯泼溅（3DGS）模型因其无需逐场景优化即可立即生成场景的能力而广受欢迎。尽管全向图像因减少了拼接以合成整体场景所需的计算而日益流行，但现有的前馈模型仅针对透视图像设计。全向图像独特的光学特性使得特征编码器难以正确理解图像上下文，并使高斯分布在空间中不均匀，这阻碍了从新视角合成的图像质量。我们提出了OmniSplat，一种无需训练、快速的前馈3DGS生成框架，专门用于全向图像。我们采用阴阳网格（Yin-Yang grid）并基于该网格分解图像，以减少全向图像与透视图像之间的域差距。阴阳网格可以原样使用现有的CNN结构，但其准均匀特性使得分解后的图像类似于透视图像，因此可以利用已学习的前馈网络的强先验知识。OmniSplat在重建精度上优于在透视图像上训练的现有前馈网络。我们的项目页面可在以下网址获取：https://robot0321.github.io/omnisplat/index.html。
英文摘要：
Feed-forward 3D Gaussian splatting (3DGS) models have gained significant popularity due to their ability to generate scenes immediately without needing per-scene optimization. Although omnidirectional images are becoming more popular since they reduce the computation required for image stitching to composite a holistic scene, existing feed-forward models are only designed for perspective images. The unique optical properties of omnidirectional images make it difficult for feature encoders to correctly understand the context of the image and make the Gaussian non-uniform in space, which hinders the image quality synthesized from novel views. We propose OmniSplat, a training-free fast feed-forward 3DGS generation framework for omnidirectional images. We adopt a Yin-Yang grid and decompose images based on it to reduce the domain gap between omnidirectional and perspective images. The Yin-Yang grid can use the existing CNN structure as it is, but its quasi-uniform characteristic allows the decomposed image to be similar to a perspective image, so it can exploit the strong prior knowledge of the learned feed-forward network. OmniSplat demonstrates higher reconstruction accuracy than existing feed-forward networks trained on perspective images. Our project page is available on: https://robot0321.github.io/omnisplat/index.html.
