arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01915cs.CVstat.CO
收藏

通过概率推断实现鲁棒逆向图形学

Robust Inverse Graphics via Probabilistic Inference

Google(谷歌公司)

Tuan Anh Le, Pavel Sountsov, Matthew D. Hoffman, Ben Lee, Brian Patton, Rif A. Saurous

AI总结:

提出贝叶斯RIG框架,利用强场景先验与均匀退化先验,从单张退化图像联合推断三维场景和退化,仅用干净数据训练即优于点估计方法。

摘要:

在雨、雪或雾等退化存在的情况下,我们如何从单张图像推断三维场景?直接的域随机化依赖于提前知道退化族。在这里,我们提出一种贝叶斯方法——称为鲁棒逆向图形学(RIG)——它依赖于强场景先验和无信息的均匀退化先验,使其适用于广泛的退化。给定单张图像,RIG对场景和退化联合执行后验推断。我们通过训练一个神经辐射场(NeRF)场景先验,并使用第二个NeRF表示退化、在其上放置无信息先验,来展示这一思想。仅在干净数据上训练的RIG优于深度估计器以及执行点估计而非完整推断的其他NeRF方法。该结果对基于归一化流和扩散模型的多种场景先验架构均成立。对于后者,我们开发了带辅助潜变量的重建引导(ReGAL)——一种扩散条件化算法,可在存在诸如退化等辅助潜变量时应用。RIG展示了场景先验如何能够用于生成任务之外。

英文摘要:

How do we infer a 3D scene from a single image in the presence of corruptions like rain, snow or fog? Straightforward domain randomization relies on knowing the family of corruptions ahead of time. Here, we propose a Bayesian approach-dubbed robust inverse graphics (RIG)-that relies on a strong scene prior and an uninformative uniform corruption prior, making it applicable to a wide range of corruptions. Given a single image, RIG performs posterior inference jointly over the scene and the corruption. We demonstrate this idea by training a neural radiance field (NeRF) scene prior and using a secondary NeRF to represent the corruptions over which we place an uninformative prior. RIG, trained only on clean data, outperforms depth estimators and alternative NeRF approaches that perform point estimation instead of full inference. The results hold for a number of scene prior architectures based on normalizing flows and diffusion models. For the latter, we develop reconstruction-guidance with auxiliary latents (ReGAL)-a diffusion conditioning algorithm that is applicable in the presence of auxiliary latent variables such as the corruption. RIG demonstrates how scene priors can be used beyond generation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑