arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.29634cs.CVcs.AI
收藏

MacTok: 图像生成中的鲁棒连续分词

MacTok: Robust Continuous Tokenization for Image Generation

Fudan University(复旦大学)

Hengyu Zeng, Xin Gao, Guanghao Li, Yuxiang Yan, Jiaoyang Ruan, Junpeng Ma, Haoyu Albert Wang, Jian Pu

AI总结:

MacTok通过引入图像掩码和表征对齐,防止后验崩溃,实现高效且高保真的连续分词,仅需64或128个token,在ImageNet上取得优异成绩。

摘要:

连续图像分词器能够高效地进行视觉生成,基于变分框架的分词器通过KL正则化学习平滑、结构化的潜在表示。然而,使用较少的token时往往导致后验崩溃,编码器无法将信息性特征编码到压缩的潜在空间中。为此,我们引入MacTok,一种通过图像掩码和表征对齐来防止崩溃的连续分词器,学习紧凑且鲁棒的表示。MacTok应用随机掩码来正则化潜在学习,并使用DINO引导的语义掩码来强调图像中的信息区域,迫使模型从不完整视觉证据中编码鲁棒的语义。结合全局和局部表征对齐,MacTok在高度压缩的1D潜在空间中保留丰富的判别信息,仅需64或128个token。在ImageNet上,MacTok在256×256时取得竞争性的gFID为1.44,在512×512时使用SiT-XL达到最先进的1.52,同时将token使用减少高达64倍。这些结果证实了掩码和语义指导共同防止后验崩溃并实现高效、高保真的分词。

英文摘要:

Continuous image tokenizers enable efficient visual generation, and those based on variational frameworks can learn smooth, structured latent representations through KL regularization. Yet this often leads to posterior collapse when using fewer tokens, where the encoder fails to encode informative features into the compressed latent space. To address this, we introduce \textbf{MacTok}, a \textbf{M}asked \textbf{A}ugmenting 1D \textbf{C}ontinuous \textbf{Tok}enizer that leverages image masking and representation alignment to prevent collapse while learning compact and robust representations. MacTok applies both random masking to regularize latent learning and DINO-guided semantic masking to emphasize informative regions in images, forcing the model to encode robust semantics from incomplete visual evidence. Combined with global and local representation alignment, MacTok preserves rich discriminative information in a highly compressed 1D latent space, requiring only 64 or 128 tokens. On ImageNet, MacTok achieves a competitive gFID of 1.44 at 256$\times$256 and a state-of-the-art 1.52 at 512$\times$512 with SiT-XL, while reducing token usage by up to 64$\times$. These results confirm that masking and semantic guidance together prevent posterior collapse and achieve efficient, high-fidelity tokenization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑