arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2606.26016cs.CV

MIMFlow:将掩码图像建模与归一化流集成用于端到端图像生成

MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Alibaba Group(阿里巴巴集团);Shanghai AI Lab(上海人工智能实验室)

Yang Chen, Xiaowei Xu, Shuai Wang, Xinwen Zhang, Qiushi Guo, Tiezheng Ge, Limin Wang

AI总结:

提出MIMFlow框架,通过VAE编码器从掩码图像推断语义潜在变量,使归一化流专注于低频语义流形,解码器处理高频合成,解决NF容量瓶颈,在ImageNet 256×256上达到71.3%线性探测精度和2.50 FID。

摘要:

归一化流(NFs)是强大的生成模型,能够进行精确的密度估计和采样。然而,其严格的可逆性常常迫使模型将容量消耗在低层像素细节上,阻碍了对高层语义结构的捕获。尽管掩码图像建模(MIM)在表示学习方面表现出色,但其在生成流程中的集成仍然主要是模块化和分离的。在本文中,我们提出了MIMFlow,一个统一的端到端框架,联合优化潜在语义、像素重建和生成流。通过采用VAE编码器从掩码图像推断语义潜在变量,MIMFlow实现了生成任务的原则性解耦:归一化流专注于建模简化的低频语义流形,而专门的解码器处理高频合成。这种设计有效地解决了NF固有的容量瓶颈,使模型能够优先考虑全局结构一致性而非冗余噪声。在ImageNet 256×256上的实验结果表明,MIMFlow-L达到了71.3%的线性探测精度和2.50的FID。尽管仅使用了128个token(比标准模型少50%),它相比类似规模的NF基线获得了32.8%的性能提升。我们的代码可在以下网址获取:https://this URL。

英文摘要:

Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low-level pixel details, hindering the capture of high-level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end-to-end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low-frequency semantic manifold, while a specialized decoder handles high-frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256$\times$256 show that MIMFlow-L reaches 71.3\% linear probing accuracy and an FID of 2.50. Despite using only 128 tokens (50\% fewer than standard models), it yields a 32.8\% performance gain over similar-scale NF baselines. Our code is available at https://github.com/MCG-NJU/MIMFlow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑