相同流，不同路径：流匹配中的方差缩减

中文摘要 AI 辅助

在流匹配（FM）中，速度模型 $v_{\theta}$ 通过预定义的路径 $g_t$ 进行训练，该路径连接数据和噪声样本（例如，$g_t(x_0, x_1) = (1 - t) x_0 + t x_1$）。在这项工作中，我们从优化角度研究该路径的选择，通过分析随机梯度的方差。我们考虑路径类别 $G(p_t,v^\star_t)$，这些路径诱导相同的边际分布 $p_t$ 和边际速度场 $v^\star_t$，因此具有相同的FM目标。我们的主要发现是，路径 $g_t$ 的选择可以从根本上改变SGD的收敛速率，即使FM目标完全相同。（i）对于线性速度模型和一维高斯数据，我们推导出SGD迭代复杂度的紧界（直至对数因子），并在诱导相同FM问题的线性路径中找到解析最优路径以最小化该界。（ii）随后，我们将方差分析扩展到一般FM问题，并在固定 $\theta$ 时将路径选择表述为方差最小化问题 PathOpt$_\theta$，约束为 $g_t\in G(p_t,v^\star_t)$。我们证明该约束至关重要：若无此约束而减少方差可能导致收敛变慢。（iii）由于约束 $g_t \in G(p_t,v^\star_t)$ 通常无法直接验证，我们推导出等价的公式，其约束可从样本中估计，从而允许数值求解路径。我们的理论结果得到了高斯数据、高斯混合模型和真实数据集的实验支持。

英文摘要

In flow matching (FM), a velocity model $v_θ$ is trained using a predefined path $g_t$ that connects data and noise samples (e.g., $g_t(x_0, x_1) = (1 - t) x_0 + t x_1$). In this work, we study the choice of this path from an optimization perspective by analyzing the variance of stochastic gradients. We consider the class $G(p_t,v^\star_t)$ of paths that induce the same marginal distributions $p_t$ and marginal velocity field $v^\star_t$, and therefore the same FM objective. Our main finding is that the choice of path $g_t$ can fundamentally change the convergence rate of SGD, even when the FM objective remains exactly the same. (i) For a linear velocity model and one-dimensional Gaussian data, we derive a tight bound on the SGD iteration complexity up to logarithmic factors and find an analytically optimal path that minimizes this bound among linear paths inducing the same FM problem. (ii) We then extend the variance analysis to general FM problems and formulate path selection at a fixed $θ$ as the variance-minimization problem PathOpt$_θ$, constrained to $g_t\in G(p_t,v^\star_t)$. We show that this constraint is essential: reducing variance without it can lead to slower convergence. (iii) Since the constraint $g_t \in G(p_t,v^\star_t)$ cannot generally be verified directly, we derive an equivalent formulation with constraints that can be estimated from samples, allowing paths to be found numerically. Our theoretical results are supported by experiments with Gaussian data, Gaussian mixture models, and real datasets.

发表机构

• Applied AI Institute（应用人工智能研究所）

机构由 AI 辅助整理，请以论文原文为准。