arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04474cs.CV

LumiGauss:野外场景下可重光照的高斯溅射

LumiGauss: Relightable Gaussian Splatting in the Wild

Warsaw University of Technology(华沙理工大学);Sano Centre for Computational Medicine(萨诺计算医学中心);IDEAS NCBR;Tooploox;Microsoft(微软)

Joanna Kaleta, Kacper Kania, Tomasz Trzcinski, Marek Kowalski

AI总结:

针对无约束照片集光照与几何解耦难、现有方法保真度低的问题,提出LumiGauss技术,基于2D高斯溅射实现场景重建与可重光照,结合球谐函数优化阴影,在NeRF-OSR数据集上性能优于基线,还可适配游戏引擎快速渲染。

摘要:

从无约束照片集中将光照与几何结构解耦是出了名的难题。解决这一问题将惠及众多用户,因为创建复杂3D资产需要数天的人工劳作。以往许多研究都尝试解决该问题,但往往以牺牲输出保真度为代价,这让此类方法的实用性存疑。我们提出LumiGauss——一种通过2D高斯溅射实现场景三维重建与环境光照解耦的技术。我们的方法能生成高质量的场景重建结果,还可在新的环境贴图下实现逼真的光照合成。我们还提出了一种利用球谐函数特性提升户外场景中常见阴影质量的方法。该方法可与游戏引擎无缝集成,还支持使用快速预计算辐射传输。我们在NeRF-OSR数据集上验证了方法,证明其性能优于基线方法。此外,LumiGauss能为未见过的环境贴图合成逼真图像。我们的代码见:https://github.com/joaxkal/lumigauss。

英文摘要:

Decoupling lighting from geometry using unconstrained photo collections is notoriously challenging. Solving it would benefit many users as creating complex 3D assets takes days of manual labor. Many previous works have attempted to address this issue, often at the expense of output fidelity, which questions the practicality of such methods. We introduce LumiGauss - a technique that tackles 3D reconstruction of scenes and environmental lighting through 2D Gaussian Splatting. Our approach yields high-quality scene reconstructions and enables realistic lighting synthesis under novel environment maps. We also propose a method for enhancing the quality of shadows, common in outdoor scenes, by exploiting spherical harmonics properties. Our approach facilitates seamless integration with game engines and enables the use of fast precomputed radiance transfer. We validate our method on the NeRF-OSR dataset, demonstrating superior performance over baseline methods. Moreover, LumiGauss can synthesize realistic images for unseen environment maps. Our code: https://github.com/joaxkal/lumigauss.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑