VENI: 自然光照的变分编码器
VENI: Variational Encoder for Natural Illumination
AI总结:
提出一种旋转等变变分自编码器,通过新型向量神经元视觉变换器(VN-ViT)编码器和旋转等变条件神经场解码器,在球面上建模自然光照,无需2D投影,实现更平滑的潜空间插值。
摘要:
逆渲染是一个不适定问题,但光照先验等先验知识有助于简化它。现有工作要么忽略了光照环境的球面和旋转等变性质,要么没有提供良好行为的潜空间。我们提出了一种旋转等变变分自编码器,无需依赖2D投影即可在球面上建模自然光照。为了保持环境图的SO(2)等变性,我们使用新型的向量神经元视觉变换器(VN-ViT)作为编码器,以及旋转等变条件神经场作为解码器。在编码器中,我们通过一种新颖的SO(2)等变全连接层(向量神经元的扩展)将等变性从SO(3)降低到SO(2)。我们展示了在我们的SO(2)等变模型中使用时,这种SO(2)等变全连接层优于标准向量神经元。与先前方法相比,我们的变分自编码器能够在潜空间中实现更平滑的插值,并提供更良好行为的潜空间。
英文摘要:
Inverse rendering is an ill-posed problem, but priors such as illumination priors can help simplify it. Existing work either disregards the spherical and rotation-equivariant nature of illumination environments or does not provide a well-behaved latent space. We propose a rotation-equivariant variational autoencoder that models natural illumination on the sphere without relying on 2D projections. To preserve the SO(2)-equivariance of environment maps, we use a novel Vector Neuron Vision Transformer (VN-ViT) as encoder and a rotation-equivariant conditional neural field as decoder. In the encoder, we reduce the equivariance from SO(3) to SO(2) using a novel SO(2)-equivariant fully connected layer, an extension of Vector Neurons. We show that our SO(2)-equivariant fully connected layer outperforms standard Vector Neurons when used in our SO(2)-equivariant model. Compared to previous methods, our variational autoencoder enables smoother interpolation in latent space and offers a more well-behaved latent space.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。