DPCS：基于路径追踪的可微投影仪-相机系统
DPCS: Path Tracing-Based Differentiable Projector-Camera Systems
AI总结：
针对神经网络式 ProCams 仿真可解释性弱、复杂光照效果差的问题，本文提出基于多反弹路径追踪和可微 PBR 的 DPCS，显式解耦并学习场景参数，以更少样本提升仿真、重光照与补偿效果。
摘要：
投影仪-相机系统（ProCams）仿真旨在对 ProCams 的物理投影与捕获过程及相关场景参数进行建模，对于空间增强现实（SAR）应用（如 ProCams 重光照和投影仪补偿）至关重要。近期进展采用端到端神经网络来学习投影与捕获过程。然而，这些基于神经网络的方法往往将表面材质、gamma 和白平衡等场景参数隐式封装在网络参数中，可解释性较差，且难以进行新场景仿真。此外，神经网络通常以图像到图像转换的方式隐式学习间接光照，导致在模拟软阴影和互反射等复杂投影效果时性能不佳。本文提出一种新颖的基于路径追踪的可微投影仪-相机系统（DPCS），提供了一种显式集成多次反弹路径追踪的可微 ProCams 仿真方法。我们的 DPCS 使用可微基于物理的渲染（PBR）对物理投影与捕获过程进行建模，使场景参数能够被显式解耦，并使用少得多的样本进行学习。此外，我们的基于物理的方法不仅能够实现高质量的下游 ProCams 任务（如 ProCams 重光照和投影仪补偿），还允许利用学习到的场景参数进行新场景仿真。在实验中，DPCS 在 ProCams 仿真方面展现出相较于先前方法的明显优势，具有更好的可解释性、对复杂互反射和阴影更高效的处理能力，并且所需训练样本更少。
英文摘要：
Projector-camera systems (ProCams) simulation aims to model the physical project-and-capture process and associated scene parameters of a ProCams, and is crucial for spatial augmented reality (SAR) applications such as ProCams relighting and projector compensation. Recent advances use an end-to-end neural network to learn the project-and-capture process. However, these neural network-based methods often implicitly encapsulate scene parameters, such as surface material, gamma, and white balance in the network parameters, and are less interpretable and hard for novel scene simulation. Moreover, neural networks usually learn the indirect illumination implicitly in an image-to-image translation way which leads to poor performance in simulating complex projection effects such as soft-shadow and interreflection. In this paper, we introduce a novel path tracing-based differentiable projector-camera systems (DPCS), offering a differentiable ProCams simulation method that explicitly integrates multi-bounce path tracing. Our DPCS models the physical project-and-capture process using differentiable physically-based rendering (PBR), enabling the scene parameters to be explicitly decoupled and learned using much fewer samples. Moreover, our physically-based method not only enables high-quality downstream ProCams tasks, such as ProCams relighting and projector compensation, but also allows novel scene simulation using the learned scene parameters. In experiments, DPCS demonstrates clear advantages over previous approaches in ProCams simulation, offering better interpretability, more efficient handling of complex interreflection and shadow, and requiring fewer training samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
