REAR：通过生成器-分词器一致性正则化重新思考视觉自回归模型
REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization
AI总结：
针对视觉自回归模型与扩散模型的性能差距，该研究发现核心瓶颈为生成器-分词器不一致，提出reAR训练策略，无需改动原有架构即可显著提升生成效果，在ImageNet上实现与大参数扩散模型相当的性能。
摘要：
视觉自回归（AR）生成为统一视觉与语言模型提供了一条颇具前景的路径，但其性能相较于扩散模型仍不够理想。此前的研究通常将这一差距归因于分词器的局限性以及光栅化顺序。在本研究中，我们从生成器-分词器不一致的角度识别出一个核心瓶颈，即AR生成的token可能无法被分词器良好解码。为解决这一问题，我们提出了reAR，这是一种引入逐token正则化目标的简单训练策略：在预测下一个token时，因果Transformer还需被训练以恢复当前token的视觉嵌入，并在含噪上下文下预测目标token的嵌入。该方法无需修改分词器、生成顺序、推理流程或外部模型。尽管方法简单，reAR却大幅提升了性能。在ImageNet数据集上，使用标准的基于光栅化的分词器时，它将gFID从3.02降至1.86，并将IS提升至316.9。当应用于先进分词器时，它仅用177M参数就实现了1.42的gFID，与参数量更大的前沿扩散模型（675M）性能相当。
英文摘要：
Visual autoregressive (AR) generation offers a promising path toward unifying vision and language models, yet its performance remains suboptimal against diffusion models. Prior work often attributes this gap to tokenizer limitations and rasterization ordering. In this work, we identify a core bottleneck from the perspective of generator-tokenizer inconsistency, i.e., the AR-generated tokens may not be well-decoded by the tokenizer. To address this, we propose reAR, a simple training strategy introducing a token-wise regularization objective: when predicting the next token, the causal transformer is also trained to recover the visual embedding of the current token and predict the embedding of the target token under a noisy context. It requires no changes to the tokenizer, generation order, inference pipeline, or external models. Despite its simplicity, reAR substantially improves performance. On ImageNet, it reduces gFID from 3.02 to 1.86 and improves IS to 316.9 using a standard rasterization-based tokenizer. When applied to advanced tokenizers, it achieves a gFID of 1.42 with only 177M parameters, matching the performance with larger state-of-the-art diffusion models (675M).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
