arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.07565cs.LGcs.AIstat.ML
收藏

归纳矩匹配

Inductive Moment Matching

Luma AI;Stanford University(斯坦福大学)

Linqi Zhou, Stefano Ermon, Jiaming Song

AI总结:

提出单阶段训练的归纳矩匹配(IMM)生成模型,支持单步或少步采样,无需预训练初始化或双网络优化,兼具稳定训练与分布收敛保证,并在 ImageNet 与 CIFAR-10 上取得领先 FID。

摘要:

扩散模型和 Flow Matching 能够生成高质量样本,但推理速度较慢;而将其蒸馏为少步模型通常会导致不稳定并需要大量调参。为解决这些权衡问题,我们提出 Inductive Moment Matching(IMM,归纳矩匹配),这是一类采用单阶段训练流程、用于单步或少步采样的新型生成模型。与蒸馏不同,IMM 不需要预训练初始化,也不需要优化两个网络;与 Consistency Models 不同,IMM 保证分布层面的收敛,并在各种超参数和标准模型架构下保持稳定。IMM 仅用 8 个推理步骤便在 ImageNet-256x256 上以 1.99 的 FID 超越扩散模型;对于从零开始训练的模型,其在 CIFAR-10 上取得 1.98 的当前最优两步 FID。

英文摘要:

Diffusion models and Flow Matching generate high-quality samples but are slow at inference, and distilling them into few-step models often leads to instability and extensive tuning. To resolve these trade-offs, we propose Inductive Moment Matching (IMM), a new class of generative models for one- or few-step sampling with a single-stage training procedure. Unlike distillation, IMM does not require pre-training initialization and optimization of two networks; and unlike Consistency Models, IMM guarantees distribution-level convergence and remains stable under various hyperparameters and standard model architectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FID using only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98 on CIFAR-10 for a model trained from scratch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑