潜空间中的物理基础渲染
Physically Based Rendering in the Latent Space
浏览论文内容
中文总结 AI 辅助
本文提出在生成模型变分自编码器学习的潜空间中执行物理基础渲染，通过修改渲染方程并结合可微渲染器，实现物理引导的内容生成，并在单张图像训练后泛化至几何、光照和视角变化。
中文摘要 AI 辅助
图像扩散模型展现了令人印象深刻的图像生成能力，但与经典计算机图形学管线（如物理基础渲染）相比，往往难以控制。然而，我们观察到光传输现象与此类模型产生的潜空间值分布之间存在一座桥梁。因此，我们引入在生成模型中变分自编码器所学习的特征空间内进行物理基础渲染，从而能够在潜空间中模拟光传输。这使我们能够利用物理基础渲染技术输出潜图，用于物理引导的内容生成。我们提出了对渲染方程的修改，当与可微渲染器配合使用时，可以产生一组最优场景参数，这些参数只需极少的细化即可准确渲染到预训练的潜空间中。我们在单张渲染图像上训练我们的方法，然后展示了该方法对场景几何变化、光照变化和相机视角变化的泛化能力。
英文摘要
Image diffusion models have shown impressive image generation capabilities but are often hard to control, in contrast to classical computer graphics pipelines such as physically based rendering. However, we observe that there is a bridge between light transport phenomena and the distribution of latent space values produced by such models. Thus, we introduce physically based rendering in the feature space learned by the variational autoencoders in generative models, enabling light transport simulation in the latent space. This allows us to leverage physically based rendering techniques to output latent maps for physically guided content generation. We propose modifications to the rendering equation, which, when paired with a differentiable renderer, can yield an optimal set of scene parameters that require only minimal refinement to accurately render into the pretrained latent space. We train our method on a single rendered image, and then demonstrate the generalization of the method to scene geometry changes, lighting changes, and camera view changes.
发表机构
- Trinity College Dublin（都柏林圣三一学院）
- École de Technologie Supérieure (ÉTS)（高等技术学院（ÉTS））
机构由 AI 辅助整理，请以论文原文为准。
