两个平静的终点与野性中间：扩散模型中记忆的几何图景
Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Models
AI总结：
本文提出了一种几何框架，通过分析噪声调度的不同区域，揭示了扩散模型中记忆与泛化的机制，并提出针对性干预以减少记忆风险。
摘要：
扩散模型能够生成高质量的样本，但也能记忆训练数据，这引发了严重的隐私问题。理解决定记忆与泛化发生机制的机制仍然是一个活跃的研究领域。特别是，不清楚在噪声调度的哪个位置会诱导记忆，数据几何如何影响它，以及不同噪声尺度的现象如何相互作用。我们引入了一个几何框架，根据训练数据由高斯壳覆盖性质和后验集中性行为将噪声调度分为三个区域，我们认为这两个基本对象是支配扩散模型中记忆和泛化的关键。这一视角揭示了记忆风险在不同噪声水平上高度非均匀。我们进一步确定了一个危险区，在中等噪声水平下记忆最明显。相比之下，小噪声和大噪声区域都抵抗记忆，但通过根本不同的机制：小噪声由于训练覆盖有限而避免记忆，而大噪声表现出后验集中性低，并且允许证明的近线性高斯去噪行为。对于中等噪声区域，我们通过几何条件提出了一个基于几何的针对性干预，以减轻记忆。
英文摘要：
Diffusion models generate high-quality samples but can also memorize training data, raising serious privacy concerns. Understanding the mechanisms governing when memorization versus generalization occurs remains an active area of research. In particular, it is unclear where along the noise schedule memorization is induced, how data geometry influences it, and how phenomena at different noise scales interact. We introduce a geometric framework that partitions the noise schedule into three regimes based on the coverage properties of training data by Gaussian shells and the concentration behavior of the posterior, which we argue are two fundamental objects governing memorization and generalization in diffusion models. This perspective reveals that memorization risk is highly non-uniform across noise levels. We further identify a danger zone at medium noise levels where memorization is most pronounced. In contrast, both the small and large noise regimes resist memorization, but through fundamentally different mechanisms: small noise avoids memorization due to limited training coverage, while large noise exhibits low posterior concentration and admits a provably near linear Gaussian denoising behavior. For the medium noise regime, we identify geometric conditions through which we propose a geometry-informed targeted intervention that mitigates memorization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
