ARSS: 通过单视角生成视图的解码器-only 自回归视觉生成的控制
ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single View
AI总结：
ARSS通过单视角生成视图，利用自回归模型和相机轨迹指导提升生成质量。
摘要：
扩散模型在世界建模任务中取得了令人印象深刻的结果，包括从稀疏输入生成新视图。然而，大多数现有的基于扩散的NVS方法通过迭代去噪过程生成目标视图，这使得在相机轨迹上施加严格因果结构变得更加困难。相比之下，自回归（AR）模型以因果方式操作，每个token都是基于所有之前生成的token生成的。在本文中，我们引入了ARSS，一种新的框架，利用GPT风格的解码器-only自回归模型，从单张图像生成新视图，并基于预定义的相机轨迹进行条件生成。我们采用现成的视频分词器将连续图像序列映射为离散token，并提出一个相机编码器，将相机轨迹转换为3D位置指导。然后，为了在保持自回归结构的同时提高生成质量，我们提出了一种自回归变压器模块，该模块在保持时间顺序的同时随机排列token的空间顺序。在公开数据集上的定性与定量实验表明，我们的方法在总体性能上与基于扩散模型的最新视图合成方法相当。项目页面：https://wbteng9526.github.io/arss/。
英文摘要：
Diffusion models have achieved impressive results in world modeling tasks, including novel view generation from sparse inputs. However, most existing diffusion-based NVS methods generate target views jointly via an iterative denoising process, which makes it less straightforward to impose a strictly causal structure along a camera trajectory. In contrast, autoregressive (AR) models operate in a causal fashion, generating each token based on all previously generated tokens. In this work, we introduce ARSS, a novel framework that leverages a GPT-style decoder-only AR model to generate novel views from a single image, conditioned on a predefined camera trajectory. We employ an off-the-shelf video tokenizer to map continuous image sequences into discrete tokens and propose a camera encoder that converts camera trajectories into 3D positional guidance. Then to enhance generation quality while preserving the autoregressive structure, we propose an autoregressive transformer module that randomly permutes the spatial order of tokens while maintaining their temporal order. Qualitative and quantitative experiments on public datasets demonstrate that our method achieves overall performance comparable to state-of-the-art view synthesis approaches based on diffusion models. Project page: https://wbteng9526.github.io/arss/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
