arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.19619stat.MLcs.LGmath.STstat.TH
收藏

ScoreFusion:通过Kullback-Leibler重心融合基于分数的生成模型

ScoreFusion: Fusing Score-based Generative Models via Kullback-Leibler Barycenters

Stanford University(斯坦福大学);HKUST(香港科技大学)

Hao Liu, Junze Tony Ye, Jose Blanchet, Nian Si

AI总结:

提出ScoreFusion方法,通过KL重心融合多预训练扩散模型,重铸为去噪扩散分数匹配以计算最优权重,在手写数字学习中验证样本效率,适配Stable Diffusion提升人像生成的群体异质性。

摘要:

我们提出ScoreFusion,这是一种理论可靠的方法,用于融合多个假设从辅助群体生成的预训练扩散模型。ScoreFusion在观测数据有限的目标群体生成建模增强方面特别有用。我们的起点考虑辅助群体的KL重心族,该族已被证明是KL意义下的最优参数类,但难以学习。不过,通过将学习问题重铸为去噪扩散中的分数匹配,我们获得了计算最优KL重心权重的可行方法。我们证明了在总变差距离下的无维度样本复杂度界,前提是辅助模型对自身任务拟合良好且辅助任务组合能很好捕捉目标。我们通过手写数字学习证明了ScoreFusion的样本效率;还提供了Stable Diffusion去噪管道的简单适配,实现从两个辅助检查点的KL重心采样,在人像生成任务中,该方法生成的人脸相对于辅助分布增强了群体异质性。

英文摘要:

We introduce ScoreFusion, a theoretically grounded method for fusing multiple pre-trained diffusion models that are assumed to generate from auxiliary populations. ScoreFusion is particularly useful for enhancing the generative modeling of a target population with limited observed data. Our starting point considers the family of KL barycenters of the auxiliary populations, which is proven to be an optimal parametric class in the KL sense, but difficult to learn. Nevertheless, by recasting the learning problem as score matching in denoising diffusion, we obtain a tractable way of computing the optimal KL barycenter weights. We prove a dimension-free sample complexity bound in total variation distance, provided that the auxiliary models are well-fitted for their own task and the auxiliary tasks combined capture the target well. The sample efficiency of ScoreFusion is demonstrated by learning handwritten digits. We also provide a simple adaptation of a Stable Diffusion denoising pipeline that enables sampling from the KL barycenter of two auxiliary checkpoints; on a portrait generation task, our method produces faces that enhance population heterogeneity relative to the auxiliary distributions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑