BEAM：弥合基于物理的渲染与高斯建模之间的鸿沟，实现可重光照的体积视频
BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Video
AI总结：
BEAM提出一种结合4D高斯表示与物理渲染的流水线，从多视角RGB视频恢复几何与PBR属性，生成可重光照体积视频，支持实时与离线渲染。
摘要：
体积视频通过捕捉动态3D场景实现沉浸式体验，为虚拟现实、教育和远程呈现等多样化应用提供了可能。然而，传统方法受限于固定光照条件，而神经方法在效率、质量或可重光照场景的适应性方面面临权衡。为解决这些局限性，我们提出了BEAM，一种新颖的流水线，将4D高斯表示与基于物理的渲染（PBR）相结合，从多视角RGB素材中生成高质量、可重光照的体积视频。BEAM通过一系列可用的基于高斯的技术恢复详细的几何形状和PBR属性。它首先在由粗到细的优化框架中，将基于高斯的人体性能跟踪与几何感知光栅化相结合，以恢复空间和时间上一致的几何形状。我们通过逐步加入PBR属性进一步增强高斯属性。我们通过多视角条件扩散模型生成粗糙度，然后利用2D到3D策略推导环境光遮蔽（AO）和基础颜色，并采用定制的基于高斯的射线追踪器进行高效的可见性计算。一旦恢复，这些动态、可重光照的资源可无缝集成到传统CG流水线中，支持使用延迟着色的实时渲染和使用光线追踪的离线渲染。通过在不同光照条件下提供逼真、栩栩如生的可视化效果，BEAM为交互式娱乐、叙事和创意可视化开辟了新的可能性。
英文摘要：
Volumetric video enables immersive experiences by capturing dynamic 3D scenes, enabling diverse applications for virtual reality, education, and telepresence. However, traditional methods struggle with fixed lighting conditions, while neural approaches face trade-offs in efficiency, quality, or adaptability for relightable scenarios. To address these limitations, we present BEAM, a novel pipeline that bridges 4D Gaussian representations with physically-based rendering (PBR) to produce high-quality, relightable volumetric videos from multi-view RGB footage. BEAM recovers detailed geometry and PBR properties via a series of available Gaussian-based techniques. It first combines Gaussian-based human performance tracking with geometry-aware rasterization in a coarse-to-fine optimization framework to recover spatially and temporally consistent geometries. We further enhance Gaussian attributes by incorporating PBR properties step by step. We generate roughness via a multi-view-conditioned diffusion model, and then derive AO and base color using a 2D-to-3D strategy, incorporating a tailored Gaussian-based ray tracer for efficient visibility computation. Once recovered, these dynamic, relightable assets integrate seamlessly into traditional CG pipelines, supporting real-time rendering with deferred shading and offline rendering with ray tracing. By offering realistic, lifelike visualizations under diverse lighting conditions, BEAM opens new possibilities for interactive entertainment, storytelling, and creative visualization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
