RenderFormer-V2：异构场景基元的神经渲染
RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives
浏览论文内容
中文总结 AI 辅助
RenderFormer-V2提出一种基于Transformer的神经渲染模型，通过两阶段序列变换和组合窗口注意力机制，支持异构场景基元，无需逐场景训练即可处理多种光传输效应。
中文摘要 AI 辅助
我们提出了“RenderFormer-V2”，一种统一的基于Transformer的学习型神经渲染模型，作为现代基于物理的渲染系统的补充，能够处理多种光传输效应，如焦散、体积散射、环境光照、带纹理和位移的表面以及分布外材质，而无需逐场景训练或专用代码。RenderFormer-V2将全局光传输建模为序列到序列的变换。沿袭其前身，RenderFormer-V2也采用两阶段流程：一个视图无关阶段，用于解析场景内基元到基元的传输；以及一个视图相关阶段，将内部神经场景表示转换为图像像素。与RenderFormer不同，我们的模型在视图无关阶段采用了一种新颖的组合窗口注意力和渲染感知注意力汇聚机制，以提高可扩展性，同时保持渲染精度。为进一步提升通用性，RenderFormer-V2支持异构场景基元，包括环境贴图和参与介质，并采用一种独立于底层表面反射模型的材质编码，通过新颖的神经嵌入来编码材质外观。我们在多种场景上展示了RenderFormer-V2的通用性，并对改进的注意力机制进行了广泛的消融研究。
英文摘要
We present 'RenderFormer-V2', a unified learned transformer-based neural rendering model, complementary to modern physics-based rendering systems, that can handle diverse light-transport effects such as caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials without per-scene training or specialized code. RenderFormer-V2 models global light transport as a sequence-to-sequence transformation. Following its predecessor, RenderFormer-V2 also employs a two stage process: a view-independent stage that resolves intra-scene primitive to primitive transport, and a view-dependent stage that transforms the internal neural scene representation into image pixels. Different from RenderFormer, our model employs a novel combined windowed-attention and rendering-informed attention sink in the view-independent stage to improve scalability while maintaining render accuracy. To further improve versatility, RenderFormerV2 supports heterogeneous scene primitives, including environment maps and participating media, and it employs a material encoding independent of the underlying surface reflectance model that encodes material appearance via a novel neural embedding. We demonstrate the versatility of RenderFormer-V2 on a variety of scenes and perform an extensive ablation of the improved attention mechanism.
发表机构
- Stanford University（斯坦福大学）
- Microsoft Research（微软研究院）
- College of William & Mary（威廉与玛丽学院）
机构由 AI 辅助整理，请以论文原文为准。
