arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09014cs.LGcs.CV
收藏

内部耦合:通过蒸馏的规范化流进行流匹配

The Coupling Within: Flow Matching via Distilled Normalizing Flows

David Berthelot, Tianrong Chen, Jiatao Gu, Marco Cuturi, Laurent Dinh, Bhavik Chandna, Michal Klein, Josh Susskind, Shuangfei Zhai

AI总结:

本文提出规范化流匹配(NFM),通过蒸馏预训练的规范化流模型的耦合来训练学生模型,从而在性能和效率上均优于传统方法。

摘要:

流模型已经迅速成为训练和部署大规模生成器的首选方法,其成功归功于在推理时通过可调节的积分步骤获得的灵活性。流训练中的关键成分是选择耦合度量来定义流匹配(FM)回归损失的采样噪声/数据对。虽然FM训练通常默认使用独立耦合,但最近的研究表明,通过噪声/数据分布(例如通过最优传输,OT)获得的自适应耦合可以提高模型训练和推理效果。我们通过这一见解进行根本性的转变:而不是直接计算自适应耦合,我们使用来自不同预训练模型的蒸馏耦合,该模型能够将噪声和数据空间置于双射中——这固有于规范化流(NF)通过其最大似然和可逆性要求。利用最近在NF图像生成中通过自回归(AR)块取得的进展,我们提出了规范化流匹配(NFM),一种新的方法,通过蒸馏预训练NF模型的近似确定性耦合来训练学生流模型。这些学生模型在两个方面都取得最佳效果:显著优于使用独立或甚至OT耦合训练的流模型,同时在教师AR-NF模型上也有所改进。

英文摘要:

Flow models have rapidly become the go-to method for training and deploying large-scale generators, owing their success to inference-time flexibility via adjustable integration steps. A crucial ingredient in flow training is the choice of coupling measure for sampling noise/data pairs that define the flow matching (FM) regression loss. While FM training defaults usually to independent coupling, recent works show that adaptive couplings informed by noise/data distributions (e.g., via optimal transport, OT) improve both model training and inference. We radicalize this insight by shifting the paradigm: rather than computing adaptive couplings directly, we use distilled couplings from a different, pretrained model capable of placing noise and data spaces in bijection -- a property intrinsic to normalizing flows (NF) through their maximum likelihood and invertibility requirements. Leveraging recent advances in NF image generation via auto-regressive (AR) blocks, we propose Normalized Flow Matching (NFM), a new method that distills the quasi-deterministic coupling of pretrained NF models to train student flow models. These students achieve the best of both worlds: significantly outperforming flow models trained with independent or even OT couplings, while also improving on the teacher AR-NF model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑