GigaTok：将视觉分词器扩展至30亿参数以用于自回归图像生成
GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation
AI总结：
针对自回归图像生成中视觉分词器扩展时重建与生成质量的矛盾，提出GigaTok方法，通过语义正则化缓解潜在空间复杂度问题，并结合三种扩展实践，将分词器扩展至30亿参数，同时提升重建、生成和表示学习性能。
摘要：
在自回归（AR）图像生成中，视觉分词器将图像压缩为紧凑的离散潜在token，通过下一个token预测实现下游自回归模型的高效训练。尽管扩展视觉分词器可提升图像重建质量，但常降低下游生成质量——现有文献未充分解决这一挑战。为此，我们提出GigaTok，这是首个在扩展视觉分词器时同时提升图像重建、生成和表示学习性能的方法。我们发现潜在空间复杂度增加是重建与生成困境的关键因素。为缓解此问题，我们提出语义正则化，将分词器特征与预训练视觉编码器的语义一致特征对齐，该约束在扩展时防止潜在空间过度复杂，同时提升重建和下游自回归生成性能。基于语义正则化，我们探索了三种扩展分词器的关键实践：（1）使用1D分词器提升可扩展性；（2）扩展编码器和解码器时优先考虑解码器扩展；（3）采用熵损失稳定十亿级参数分词器的训练。扩展至30亿参数后，GigaTok在重建、下游AR生成和下游AR表示质量上均达到最先进性能。
英文摘要：
In autoregressive (AR) image generation, visual tokenizers compress images into compact discrete latent tokens, enabling efficient training of downstream autoregressive models for visual generation via next-token prediction. While scaling visual tokenizers improves image reconstruction quality, it often degrades downstream generation quality -- a challenge not adequately addressed in existing literature. To address this, we introduce GigaTok, the first approach to simultaneously improve image reconstruction, generation, and representation learning when scaling visual tokenizers. We identify the growing complexity of latent space as the key factor behind the reconstruction vs. generation dilemma. To mitigate this, we propose semantic regularization, which aligns tokenizer features with semantically consistent features from a pre-trained visual encoder. This constraint prevents excessive latent space complexity during scaling, yielding consistent improvements in both reconstruction and downstream autoregressive generation. Building on semantic regularization, we explore three key practices for scaling tokenizers:(1) using 1D tokenizers for better scalability, (2) prioritizing decoder scaling when expanding both encoder and decoder, and (3) employing entropy loss to stabilize training for billion-scale tokenizers. By scaling to $\bf{3 \space billion}$ parameters, GigaTok achieves state-of-the-art performance in reconstruction, downstream AR generation, and downstream AR representation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
