控制隐式生成模型中的依赖性：通过扩散互信息

中文摘要 AI 辅助

互信息（MI）为在隐式生成模型中抑制或鼓励统计依赖性提供了一种目标函数。然而，在隐式模型中直接评估MI具有挑战性，因为其密度通常难以处理。一种补救方法是通过条件得分与边际得分之差来估计生成器梯度。这种得分差又可以通过对通过分类学习到的对数密度比进行微分来估计。然而，这种构造面临两个困难：（i）奇异分布可能不承认所需的得分函数，以及（ii）较差的重叠可能阻碍密度比估计。因此，我们引入了扩散互信息（SMI），这是通过将共同的扩散核应用于生成变量，在噪声水平上对MI进行加权积分。高斯扩散产生平滑、严格正的条件和边际密度，将梯度构造扩展到可能原本是奇异的分布。在多种实验中，SMI在基于MI的方法中始终实现有效的依赖性控制，并与已建立的任务特定方法保持竞争力。

英文摘要

Mutual information (MI) provides an objective for suppressing or encouraging statistical dependence in implicit generative models. However, direct MI evaluation is challenging in implicit models due to typically intractable densities. A remedy is estimating the generator gradient from the difference between conditional and marginal scores. This score difference can, in turn, be estimated by differentiating a log density ratio learned through classification. This construction nevertheless faces two difficulties: (i) singular distributions need not admit the required score functions, and (ii) poor overlap can hinder density-ratio estimation. We therefore introduce Spread Mutual Information (SMI), a weighted integral of MI across noise levels obtained by applying a common spreading kernel to the generated variable. Gaussian spreading yields smooth, strictly positive conditional and marginal densities, extending the gradient construction to distributions that may originally be singular. Across a variaty of experiments, SMI consistently achieves effective dependence control among MI-based methods and remains competitive with established task-specific approaches.

发表机构

• University of Oxford（牛津大学）
• University of Bristol（布里斯托尔大学）
• University of Cambridge（剑桥大学）

机构由 AI 辅助整理，请以论文原文为准。