arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.02174stat.MLcs.AIcs.LG
收藏

扩散模型的训练数据归因

Training Data Attribution for Diffusion Models

Massachusetts Institute of Technology(麻省理工学院)

Zheng Dai, David K Gifford

AI总结:

针对扩散模型难以追溯训练数据影响的问题,提出在精心划分的数据上训练编码集成,以识别关键样本并消融其影响,验证了方法有效性。

摘要:

扩散模型已越来越多地用于基于训练数据集合成高质量样本。然而,鉴于训练数据集通常规模庞大,评估训练数据如何影响训练后的扩散模型所生成的样本十分困难。将扩散模型输入与输出关联起来的难度,给模型可解释性和训练数据归因带来了重大挑战。为此,我们提出了一种新颖的解决方案,通过使用集成模型揭示训练数据如何影响扩散模型的输出。在我们的方法中,编码集成中的各个模型在整体训练数据经过精心设计的划分上进行训练,从而能够识别具有影响力的训练样本。所得的模型集成支持高效地消融训练数据影响,使我们能够评估训练数据对模型输出的影响。我们证明了这些集成作为生成模型的可行性,以及我们评估影响的方法的有效性。

英文摘要:

Diffusion models have become increasingly popular for synthesizing high-quality samples based on training datasets. However, given the oftentimes enormous sizes of the training datasets, it is difficult to assess how training data impact the samples produced by a trained diffusion model. The difficulty of relating diffusion model inputs and outputs poses significant challenges to model explainability and training data attribution. Here we propose a novel solution that reveals how training data influence the output of diffusion models through the use of ensembles. In our approach individual models in an encoded ensemble are trained on carefully engineered splits of the overall training data to permit the identification of influential training examples. The resulting model ensembles enable efficient ablation of training data influence, allowing us to assess the impact of training data on model outputs. We demonstrate the viability of these ensembles as generative models and the validity of our approach to assessing influence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑