ERUPT：基于无位姿图像块Transformer的高效渲染
ERUPT: Efficient Rendering with Unposed Patch Transformer
AI总结：
提出ERUPT模型，通过基于图像块的查询和学习潜在相机位姿，仅需少量无位姿图像即可高效合成新视角，显著降低计算与数据需求并提升渲染质量。
摘要：
本工作解决了从小批量RGB图像集合中进行多样化场景新视角合成的问题。我们提出了ERUPT（Efficient Rendering with Unposed Patch Transformer），这是一种最先进的场景重建模型，能够使用无位姿图像进行高效场景渲染。与现有的基于像素的查询不同，我们引入了基于图像块的查询，以减少渲染目标视角所需的计算量。这使得我们的模型在训练和推理期间都非常高效，能够在商用硬件上以600 fps的速度进行渲染。值得注意的是，我们的模型设计为使用学习到的潜在相机位姿，允许在具有稀疏或不准确真实相机位姿的数据集中使用无位姿目标进行训练。我们证明了我们的方法可以在大型真实世界数据上泛化，并引入了一个新的基准数据集（MSVS-1M），用于使用从Mapillary收集的街景图像进行潜在视角合成。与需要密集图像和精确元数据的NeRF和Gaussian Splatting不同，ERUPT可以仅用五张无位姿输入图像渲染任意场景的新视角。ERUPT在无位姿图像合成任务上实现了比当前最先进方法更好的渲染图像质量，将标记数据需求减少了约95%，并将计算需求降低了一个数量级，为多样化的真实世界场景提供了高效的新视角合成。
英文摘要：
This work addresses the problem of novel view synthesis in diverse scenes from small collections of RGB images. We propose ERUPT (Efficient Rendering with Unposed Patch Transformer) a state-of-the-art scene reconstruction model capable of efficient scene rendering using unposed imagery. We introduce patch-based querying, in contrast to existing pixel-based queries, to reduce the compute required to render a target view. This makes our model highly efficient both during training and at inference, capable of rendering at 600 fps on commercial hardware. Notably, our model is designed to use a learned latent camera pose which allows for training using unposed targets in datasets with sparse or inaccurate ground truth camera pose. We show that our approach can generalize on large real-world data and introduce a new benchmark dataset (MSVS-1M) for latent view synthesis using street-view imagery collected from Mapillary. In contrast to NeRF and Gaussian Splatting, which require dense imagery and precise metadata, ERUPT can render novel views of arbitrary scenes with as few as five unposed input images. ERUPT achieves better rendered image quality than current state-of-the-art methods for unposed image synthesis tasks, reduces labeled data requirements by ~95\% and decreases computational requirements by an order of magnitude, providing efficient novel view synthesis for diverse real-world scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
