发表机构
Johns Hopkins University（约翰霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对离散随机变量，提出有限阶精确路径梯度估计器，无偏、低方差、无需调温，并在实验中匹配或超越基线且运行更快。
AI 中文摘要
路径梯度因无偏、低方差且只需单个样本即可工作，而成为连续随机变量的首选。然而，对于离散变量，路径恒等式通常无法对每个可微函数都精确成立。我们提出了一个通用框架，为一系列常见的离散变量（如泊松变量）构建有限阶精确路径梯度估计器。该估计器是所有对至多给定次数的多项式无偏的解中范数最小的解。由此产生的估计器保留了硬前向样本，无需调节温度，并且只需几行代码即可实现。与其他可行解相比，我们的估计器是唯一的，并且最小化了权重方差；相比之下，先前的工作使用分类变量或增广表示来近似非分类变量，这引入了额外的方差和计算量。为了理解超出规定类别函数的近似偏差，我们还推导了一个非渐近偏差界。在实验中，我们的低阶方法在线性、非线性和分层潜变量模型上匹配或改进了调优基线，并在每个运行时基准测试中都比竞争对手更快。
英文摘要
Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete variables, however, the pathwise identity cannot generally be exact for every differentiable function. We propose a general framework to construct finite-order exact pathwise gradient estimators for a range of common discrete variables such as Poisson. The estimator is the least-norm solution among all solutions that are unbiased for polynomials of degree at most. The resulting estimators preserve the hard forward sample, require no temperature tuning, and can be implemented in a few lines of codes. Against other admissible solutions, our estimator is unique and minimizes weight variance; in contrast, prior works use categorical variables or augmented representations to approximate non-categorical variables that induces excess variance and computations. To understand approximation bias for functions beyond the prescribed class, we also derive a non-asymptotic bias bound. In experiments our low order methods match or improve tuned baselines across linear, nonlinear and hierarchical latent-variable models, while out-speeding competitors in every runtime benchmark.
