arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.00802cs.LGcs.AI
收藏

Concrete Score Matching:离散数据的广义分数匹配

Concrete Score Matching: Generalized Score Matching for Discrete Data

Stanford University(斯坦福大学);NVIDIA(英伟达);CZ Biohub(陈·扎克伯格生物中心)

Chenlin Meng, Kristy Choi, Jiaming Song, Stefano Ermon

摘要:

通过密度函数的梯度表示概率分布已被证明在对各种连续数据模态进行建模时是有效的。然而,这种表示法不适用于梯度未定义的离散域。为此,我们提出了一个类似的分数函数称为“具体的分数”,概括的(斯坦)分数离散设置。给定一个预定义的邻域结构,任何输入的具体分数都由概率相对于输入的局部方向变化的变化率定义。这个公式使我们能够恢复(斯坦)分数在连续域测量这种变化时,通过欧几里德距离,而使用曼哈顿距离导致我们的新的分数函数在离散域。最后,我们引入了一个新的框架来从样本中学习这样的分数,称为混凝土分数匹配(CSM),并提出了一个有效的训练目标来扩展我们的方法到高维。从经验上讲,我们证明了CSM对合成,表格和高维图像数据集的混合物上的密度估计任务的有效性,并证明了它相对于现有的离散数据建模基线表现良好。

英文摘要:

Representing probability distributions by the gradient of their density functions has proven effective in modeling a wide range of continuous data modalities. However, this representation is not applicable in discrete domains where the gradient is undefined. To this end, we propose an analogous score function called the "Concrete score", a generalization of the (Stein) score for discrete settings. Given a predefined neighborhood structure, the Concrete score of any input is defined by the rate of change of the probabilities with respect to local directional changes of the input. This formulation allows us to recover the (Stein) score in continuous domains when measuring such changes by the Euclidean distance, while using the Manhattan distance leads to our novel score function in discrete domains. Finally, we introduce a new framework to learn such scores from samples called Concrete Score Matching (CSM), and propose an efficient training objective to scale our approach to high dimensions. Empirically, we demonstrate the efficacy of CSM on density estimation tasks on a mixture of synthetic, tabular, and high-dimensional image datasets, and demonstrate that it performs favorably relative to existing baselines for modeling discrete data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑