arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.07741stat.MLcs.LG
收藏

扩散模型的高斯普适性

Gaussian Universality for Diffusion Models

Department of Electrical Engineering, California Institute of Technology(电气工程系,加利福尼亚理工学院);Department of Electrical and Computer Engineering, University of Texas at Austin(电气与计算机工程系,德克萨斯大学奥斯汀分校);Department of Computing and Mathematical Sciences, California Institute of Technology(计算与数学科学系,加利福尼亚理工学院)

Reza Ghane, Anthony Bao, Danil Akhtiamov, Babak Hassibi

AI总结:

本文研究扩散生成数据的高斯普适性,证明线性分类器测试误差仅由一阶、二阶统计量决定,并给出条件扩散样本的 Lipschitz 函数集中结论。

摘要:

我们研究由扩散模型生成的数据分布的高斯普适性。所谓高斯普适性,是指在扩散数据上为分类任务训练的广义线性模型 $f(\mathbf{W})$ 的测试误差,与在各类别均值和协方差相匹配的高斯混合模型上训练的 $f(\mathbf{W})$ 的测试误差一致。换言之,在线性设定下,测试误差仅取决于扩散生成数据的一阶和二阶统计量。作为推论,线性分类器测试误差的分析可从扩散生成数据归约为高斯数据。由于诸如 \cite{sehwag2024stretchingdollardiffusiontraining} 等方法的兴起,分析在合成数据上训练的模型性能是一个切题问题。此外,我们证明,对于任意 1-Lipschitz 标量函数 $ϕ$,当 $\mathbf{x}$ 从对应每个类别的条件扩散模型中采样时,$ϕ(\mathbf{x})$ 以高概率接近 $\mathbb{E} ϕ(\mathbf{x})$。最后,我们指出,当前用于证明普适性的方法不适用于扩散生成数据,因为数据协方差矩阵的最小奇异值往往趋于零,这与文献中的假设相反。这使得扩展以往的数学普适性结果成为一个引人关注的开放问题。

英文摘要:

We investigate Gaussian Universality for data distributions generated via diffusion models. By Gaussian Universality we mean that the test error of a generalized linear model $f(\mathbf{W})$ trained for a classification task on the diffusion data matches the test error of $f(\mathbf{W})$ trained on the Gaussian Mixture with matching means and covariances per class.In other words, the test error depends only on the first and second order statistics of the diffusion-generated data in the linear setting. As a corollary, the analysis of the test error for linear classifiers can be reduced to Gaussian data from diffusion-generated data. Analysing the performance of models trained on synthetic data is a pertinent problem due to the surge of methods such as \cite{sehwag2024stretchingdollardiffusiontraining}. Moreover, we show that, for any $1$- Lipschitz scalar function $ϕ$, $ϕ(\mathbf{x})$ is close to $\mathbb{E} ϕ(\mathbf{x})$ with high probability for $\mathbf{x}$ sampled from the conditional diffusion model corresponding to each class. Finally, we note that current approaches for proving universality do not apply to diffusion-generated data as the covariance matrices of the data tend to have vanishing minimum singular values, contrary to the assumption made in the literature. This leaves extending previous mathematical universality results as an intriguing open question.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑