arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.02548stat.MLcs.LG
收藏

局部流匹配生成模型

Local Flow Matching Generative Models

H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology(哈里·米尔顿·斯图尔特工业与系统工程学院,佐治亚理工学院);Department of Mathematics, Duke University(杜克大学数学系)

Chen Xu, Xiuyuan Cheng, Yao Xie

AI总结:

提出局部流匹配(LFM)方法,通过逐步学习一系列流匹配子模型,每个子模型匹配扩散过程的一个时间步,实现更高效训练,并基于χ²散度提供理论生成保证,在表格、图像和机器人策略生成中表现竞争性。

摘要:

流匹配(FM)是一种无模拟方法,用于学习连续、可逆的流,该流在两个分布之间插值,特别是从噪声生成数据。受扩散过程作为梯度流的变分性质启发,我们引入了一种逐步FM模型,即局部流匹配(LFM),它顺序学习一系列FM子模型,每个子模型匹配数据到噪声方向上的扩散过程直至时间步长。在每一步中,子流模型要插值的两个分布比完整流匹配模型(插值数据到噪声分布)中的更接近,从而允许使用更小的模型进行更高效的训练。这种变分视角还使我们能够利用扩散过程的收缩性质,为所提出的流模型提供基于χ²散度的理论生成保证。在实践中,LFM的逐步结构自然适用于模型蒸馏,并且可以应用各种蒸馏技术来加速生成。我们通过实验证明,在表格和图像数据集的无条件生成以及机器人操作策略的条件生成上,LFM相比FM实现了具有竞争力的生成性能。

英文摘要:

Flow Matching (FM) is a simulation-free method for learning a continuous, invertible flow that interpolates between two distributions, and in particular generates data from noise. Inspired by the variational nature of the diffusion process as a gradient flow, we introduce a stepwise FM model, Local Flow Matching (LFM), which sequentially learns a sequence of FM submodels, each matching a diffusion process up to the time-step size in the data-to-noise direction. In each step, the two distributions to be interpolated by the sub-flow model are closer than those in the full-flow matching model, which interpolates data to noise distributions, enabling smaller models with more efficient training. This variational perspective also allows us to prove a theoretical generation guarantee for the proposed flow model in terms of the $χ^2$-divergence between the generated and true data distributions, leveraging the contraction property of the diffusion process. In practice, the stepwise structure of LFM is naturally amenable to model distillation, and various distillation techniques can be applied to accelerate generation. We empirically demonstrate that LFM achieves competitive generative performance compared to FM on unconditional generation of tabular and image datasets, and on conditional generation of robotic manipulation policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑