Mirai: 自回归视觉生成需要前瞻性
Mirai: Autoregressive Visual Generation Needs Foresight
AI总结：
研究探讨前瞻性训练信号对自回归视觉生成的影响，提出Mirai框架通过注入未来信息提升生成质量与收敛速度。
摘要：
自回归（AR）视觉生成器将图像建模为离散令牌序列，并通过下一个令牌似然目标进行训练。这种严格的因果监督仅基于立即下一个令牌优化每一步，可能削弱全局一致性并减缓收敛。本文研究前瞻性（训练信号源自后续令牌）能否提升自回归视觉生成。通过一系列受控诊断，揭示关键见解：将前瞻性对齐AR模型的内部表示可提升因果建模。提出Mirai框架，无需架构改动和额外推理开销，通过显式和隐式前瞻性信息提升生成质量。实验显示，Mirai显著加速收敛并提升生成质量，例如将LlamaGen-B的收敛速度提升10倍，将ImageNet类条件图像生成基准的FID从5.34降至4.34。研究强调视觉自回归模型需要前瞻性。
英文摘要：
Autoregressive (AR) visual generators model images as sequences of discrete tokens and are trained with a next-token likelihood objective. This strict causal supervision optimizes each step based only on the immediate next token, which can weaken global coherence and slow convergence. We investigate whether foresight, training signals that originate from later tokens, can improve autoregressive visual generation. We conduct a series of controlled diagnostics along the injection level, foresight layout, and foresight source axes, revealing a key insight: aligning foresight with AR models' internal representations on the 2D image grid improves causal modeling. We formulate this insight with Mirai (meaning "future" in Japanese), a general framework that injects future information into AR training with no architecture change and no extra inference overhead: Mirai-E uses explicit foresight from multiple future positions of unidirectional representations, whereas Mirai-I leverages implicit foresight from matched bidirectional representations. Extensive experiments show that Mirai significantly accelerates convergence and improves generation quality. For instance, Mirai can speed up LlamaGen-B's convergence by up to 10$\times$ and reduce the generation FID from 5.34 to 4.34 on the ImageNet class-condition image generation benchmark. Our study highlights that visual autoregressive models need foresight.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
