自回归视觉生成需要一个序幕
Autoregressive Visual Generation Needs a Prologue
AI总结：
提出Prologue方法，通过生成前置的序幕令牌来弥合自回归图像生成中的重建-生成差距，在不影响重建质量的前提下显著提升生成性能。
摘要：
在这项工作中，我们提出了Prologue，一种弥合自回归（AR）图像生成中重建-生成差距的方法。Prologue不修改视觉令牌以同时满足重建和生成，而是生成一小部分序幕令牌，并将其前置到视觉令牌序列之前。这些序幕令牌仅使用AR交叉熵（CE）损失进行训练，而视觉令牌则专用于重建。这种解耦设计使我们能够通过AR模型的真实分布优化生成，而不影响重建质量，我们进一步从ELBO角度形式化了这一点。在ImageNet 256x256上，Prologue-Base在没有无分类器引导的情况下将gFID从21.01降至10.75，同时几乎保持重建不变；Prologue-Large使用标准AR模型，无需辅助语义监督，达到了具有竞争力的rFID 0.99和gFID 1.46。有趣的是，仅由AR梯度驱动，序幕令牌展现出涌现的语义结构：对16个序幕令牌进行线性探测达到35.88%的Top-1准确率，远高于标准分词器前16个令牌的23.71%；使用固定序幕令牌进行重采样保留了相似的高层语义布局。我们的结果暗示了一个新方向：通过引入单独学习的生成表示，同时保持原始表示不变，可以提升生成质量。
英文摘要：
In this work, we propose Prologue, an approach to bridging the reconstruction-generation gap in autoregressive (AR) image generation. Instead of modifying visual tokens to satisfy both reconstruction and generation, Prologue generates a small set of prologue tokens prepended to the visual token sequence. These prologue tokens are trained exclusively with the AR cross-entropy (CE) loss, while visual tokens remain dedicated to reconstruction. This decoupled design lets us optimize generation through the AR model's true distribution without affecting reconstruction quality, which we further formalize from an ELBO perspective. On ImageNet 256x256, Prologue-Base reduces gFID from 21.01 to 10.75 without classifier-free guidance while keeping reconstruction almost unchanged; Prologue-Large reaches a competitive rFID of 0.99 and gFID of 1.46 using a standard AR model without auxiliary semantic supervision. Interestingly, driven only by AR gradients, prologue tokens exhibit emergent semantic structure: linear probing on 16 prologue tokens reaches 35.88% Top-1, far above the 23.71% of the first 16 tokens from a standard tokenizer; resampling with fixed prologue tokens preserves a similar high-level semantic layout. Our results suggest a new direction: generation quality can be improved by introducing a separate learned generative representation while leaving the original representation intact.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
