arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.18593cs.CVcs.AIcs.LG
收藏

扩散自编码器是可扩展的图像分词器

Diffusion Autoencoders are Scalable Image Tokenizers

Meta;GenAI, Meta(元公司生成式人工智能部门)

Yinbo Chen, Rohit Girdhar, Xiaolong Wang, Sai Saketh Rambhatla, Ishan Misra

AI总结:

提出扩散分词器DiTo,利用单一扩散L2损失训练可扩展图像分词器,简化训练流程,在图像重建与生成任务中达到与有监督最先进方法相当或更优的质量。

摘要:

将图像分词为紧凑的视觉表示是学习高效且高质量图像生成模型的关键步骤。我们提出了一种简单的扩散分词器(DiTo),用于为图像生成模型学习紧凑的视觉表示。我们的核心见解是,单一的学习目标,即扩散L2损失,可用于训练可扩展的图像分词器。由于扩散已被广泛用于图像生成,我们的见解极大地简化了此类分词器的训练。相比之下,当前最先进的分词器依赖于经验性发现的启发式方法和损失组合,因此需要复杂的训练配方,依赖于非平凡地平衡不同损失和预训练的监督模型。我们展示了设计决策及其理论基础,使我们能够扩展DiTo以学习具有竞争力的图像表示。我们的结果表明,DiTo是当前最先进的有监督图像分词器的一种更简单、可扩展且自监督的替代方案。在图像重建和下游图像生成任务中,DiTo实现了与最先进方法相当或更好的质量。

英文摘要:

Tokenizing images into compact visual representations is a key step in learning efficient and high-quality image generative models. We present a simple diffusion tokenizer (DiTo) that learns compact visual representations for image generation models. Our key insight is that a single learning objective, diffusion L2 loss, can be used for training scalable image tokenizers. Since diffusion is already widely used for image generation, our insight greatly simplifies training such tokenizers. In contrast, current state-of-the-art tokenizers rely on an empirically found combination of heuristics and losses, thus requiring a complex training recipe that relies on non-trivially balancing different losses and pretrained supervised models. We show design decisions, along with theoretical grounding, that enable us to scale DiTo for learning competitive image representations. Our results show that DiTo is a simpler, scalable, and self-supervised alternative to the current state-of-the-art image tokenizer which is supervised. DiTo achieves competitive or better quality than state-of-the-art in image reconstruction and downstream image generation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑