粘性跳跃扩散：掩码、连续和混合扩散的统一视角

中文摘要 AI 辅助

我们引入了粘性跳跃扩散（SJDs），它是在\(\mathbb{R}^d\)上的连续时间马尔可夫过程，其离散锚点是令牌嵌入。在正向时间里，锚点以危险率释放其质量，释放的质量在连续环境空间中扩散；时间反转将一个由分数驱动的随机微分方程与一个粘性跳跃核耦合，其速率和目的地由与正向定律的通量平衡确定。我们通过去噪危险匹配从单个去噪分类器估计分数和每个锚点的反向危险，这是去噪分数匹配的危险类似物，采用无模拟交叉熵训练。SJD将掩码扩散、连续扩散和混合扩散作为极限恢复。其反转解释了每个类别视为给定的特征：掩码扩散的掩码没有携带关于源令牌的证据，因为每个锚点的解粘核坍缩到同一个吸收点；连续扩散的终端投影是必需的，因为其正向边缘没有原子，没有它通量平衡不会产生反向跳跃；混合扩散的更新规则（提交率、目的地和漂移）都来自通量平衡而不是单独设计。除了这些极限，解粘核成为一个设计空间：跨位置混合将每个位置朝着其邻居的干净值或嵌入的混合方向损坏，将诸如空间局部性或约束图等依赖结构转化为损坏本身的归纳偏差，并在CIFAR - 10、Text8和数独上优于恒等核混合方法。

英文摘要

We introduce Sticky Jump Diffusions (SJDs), continuous-time Markov processes on $\mathbb R^d$ whose discrete anchors are token embeddings. In forward time, anchors release their mass at a hazard rate and the released mass diffuses in the continuous ambient space; time reversal couples a score-driven SDE with a sticky jump kernel whose rate and destination are fixed by flux balance with the forward law. We estimate the score and the per-anchor reverse hazards from a single denoising classifier via Denoising Hazard Matching, the hazard analogue of denoising score matching, with simulation-free cross-entropy training. SJD recovers masked diffusion, continuous diffusion, and hybrid diffusion as limits. Its reversal explains features that each family treats as given: the mask of masked diffusion carries no evidence about the source token because the unsticking kernel of every anchor collapses to the same absorbing point; the terminal projection of continuous diffusion is required due to the absence of atoms in its forward marginal, without which flux balance yields no reverse jumps; and the update rules of hybrid diffusion (commit rate, destination, and drift) all follow from flux balance rather than from separate design. Beyond these limits, the unsticking kernel becomes a design space: a cross-position blending corrupts each position toward a blend of its neighbors' clean values or embeddings, turning dependency structure such as spatial locality or a constraint graph into an inductive bias of the corruption itself, and improves over the identity-kernel hybrid on CIFAR-10, Text8, and Sudoku.

发表机构

• Purdue University（普渡大学）

机构由 AI 辅助整理，请以论文原文为准。