UMAMI：统一掩码自回归模型和确定性渲染用于视图合成
UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis
AI总结：
UMAMI通过结合掩码自回归模型和确定性渲染，实现了视图合成中图像质量与渲染效率的统一。
摘要：
新颖的视图合成（NVS）旨在从未见的摄像机姿态下，仅给定稀疏的摄像机视角，渲染出逼真且三维一致的图像。现有确定性网络能够快速渲染观测区域，但会模糊未观测区域，而随机扩散方法虽然能够生成合理的图像内容，但会带来高昂的训练和推理时间成本。在本文中，我们提出了一种混合框架，将两种方法的优势相结合。双向变压器编码多视角图像标记和普鲁克射线嵌入，生成共享的潜在表示。两个轻量级头部则对这一表示进行操作：（i）一个前馈回归头部用于渲染几何约束良好的像素区域；（ii）一个掩码自回归扩散头部用于完成被遮挡或未观测的区域。整个模型通过联合光度和扩散损失端到端训练，不依赖手工制作的3D归纳偏置，使模型能够适用于多样化的场景。实验表明，我们的方法在图像质量上达到当前最先进的水平，同时将渲染时间比完全生成基线减少了数量级。
英文摘要：
Novel view synthesis (NVS) seeks to render photorealistic, 3D-consistent images of a scene from unseen camera poses given only a sparse set of posed views. Existing deterministic networks render observed regions quickly but blur unobserved areas, whereas stochastic diffusion-based methods hallucinate plausible content yet incur heavy training- and inference-time costs. In this paper, we propose a hybrid framework that unifies the strengths of both paradigms. A bidirectional transformer encodes multi-view image tokens and Plucker-ray embeddings, producing a shared latent representation. Two lightweight heads then act on this representation: (i) a feed-forward regression head that renders pixels where geometry is well constrained, and (ii) a masked autoregressive diffusion head that completes occluded or unseen regions. The entire model is trained end-to-end with joint photometric and diffusion losses, without handcrafted 3D inductive biases, enabling scalability across diverse scenes. Experiments demonstrate that our method attains state-of-the-art image quality while reducing rendering time by an order of magnitude compared with fully generative baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
