arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.05930cs.LGcs.AImath.DG
收藏

Carré du champ flow matching: 更好的质量-泛化权衡在生成模型中

Carré du champ flow matching: better quality-generalisation tradeoff in generative models

Institute of Artificial Intelligence(人工智能研究所);Medical University of Vienna(维也纳医科大学);University of Oxford(牛津大学);AITHYRA;EPFL(苏黎世联邦理工学院)

Jacob Bamberger, Iolo Jones, Dennis Duncan, Michael M. Bronstein, Pierre Vandergheynst, Adam Gosztolai

AI总结:

Carré du champ flow matching通过几何感知噪声改进生成模型的质量-泛化权衡,适用于数据稀缺和非均匀采样场景。

摘要:

深度生成模型常常面临一个根本性的权衡:高质量的样本可以以牺牲记忆为代价,其中模型复制训练数据而不是在数据几何上泛化。我们引入了Carré du champ flow matching(CDC-FM),这是flow matching(FM)的一种扩展,通过使用几何感知的噪声来正则化概率路径,从而改进质量-泛化权衡。我们的方法用空间变化、各向异性的高斯噪声替代了FM中的同质、各向同性的噪声,其协方差捕捉了潜在数据流形的局部几何。我们证明这种几何噪声可以从数据中最优估计,并且可以扩展到大规模数据。进一步,我们在多样化的数据集(合成流形、点云、单细胞基因组学、动物动作捕捉和图像)以及各种神经网络架构(MLPs、CNNs和transformers)上进行了广泛的实验评估。我们证明CDC-FM在质量-泛化权衡上始终表现更好。我们观察到在数据稀缺的领域和高度非均匀采样的数据集中,标准FM有显著的改进,这些常常出现在AI for science应用中。我们的工作提供了一个数学框架,用于研究生成模型中数据几何、泛化和记忆之间的相互作用,以及一个稳健且可扩展的算法,可以轻松地集成到现有的flow matching流程中。

英文摘要:

Deep generative models often face a fundamental tradeoff: high sample quality can come at the cost of memorisation, where the model reproduces training data rather than generalising across the underlying data geometry. We introduce Carré du champ flow matching (CDC-FM), a generalisation of flow matching (FM), that improves the quality-generalisation tradeoff by regularising the probability path with a geometry-aware noise. Our method replaces the homogeneous, isotropic noise in FM with a spatially varying, anisotropic Gaussian noise whose covariance captures the local geometry of the latent data manifold. We prove that this geometric noise can be optimally estimated from the data and is scalable to large data. Further, we provide an extensive experimental evaluation on diverse datasets (synthetic manifolds, point clouds, single-cell genomics, animal motion capture, and images) as well as various neural network architectures (MLPs, CNNs, and transformers). We demonstrate that CDC-FM consistently offers a better quality-generalisation tradeoff. We observe significant improvements over standard FM in data-scarce regimes and in highly non-uniformly sampled datasets, which are often encountered in AI for science applications. Our work provides a mathematical framework for studying the interplay between data geometry, generalisation and memorisation in generative models, as well as a robust and scalable algorithm that can be readily integrated into existing flow matching pipelines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑