发表机构
Instituto de Informática (INF); Universidade Federal de Goiás (UFG)（信息学研究所; 戈亚斯联邦大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文证明总变差而非KL散度能直接预测压缩语言模型的决策翻转率，实验验证其比率稳定且无需校准，优于KL散度。
AI 中文摘要
压缩报告通常通过KL散度总结压缩语言模型与稠密模型之间的移动距离；依赖稠密模型输出的部署需要知道其决策改变了多少。我们证明，总变差而非KL散度能直接回答这一问题。在19个开放模型的802个压缩和扰动副本上，跨越五个语料库和九个机械无关的扰动族，arg-max令牌变化的速率（即翻转率）以中位数1.05的比率追踪总变差，且无需拟合常数。KL散度仅通过其平方根以及一个在模型和语料库间变化四倍的因子转化为翻转，因为KL在取根之前对令牌进行平均；每令牌平均的一阶统计量（如Hellinger距离）避免了这一问题，但报告很少提供这些统计量。因此，在两个报告于不同模型和语料库的压缩器中，若其翻转率相差至少10%，KL在11%的情况下将较小的散度分配给决策变化更多的那个，而总变差在1%的情况下如此。两项预注册测试标明了限制：在保留的代码语料库上，比率对所有八个模型成立，而关于KL的三个预测中每个至少对一半模型失败；在三个具有真实内核的新模型上，比率在38个检查点中的37个保持在其区间内，但在代码上对两个模型降至1以下。在vLLM投机解码中，在教师强制下测量的总变差以平均相对误差1.1%–2.4%预测贪婪草稿接受率，无需KL所需的特定任务校准。
英文摘要
Compression reports summarize how far a compressed language model moved from the dense one, usually by a KL divergence; a deployment that relies on the dense model's outputs needs to know how many of its decisions changed. We show that total variation, not KL, answers this directly. Across 802 compressed and perturbed copies of 19 open models on five corpora and nine mechanically unrelated perturbation families, the rate at which the arg-max token changes (the \emph{flip rate}) tracks total variation at a ratio with median $1.05$, with no fitted constant. KL converts into flips only through its square root and a factor that varies fourfold across models and corpora, because KL averages over tokens before the root is taken; first-order statistics averaged per token, such as Hellinger distance, avoid this, but reports rarely give them. As a result, of two compressors reported on different models and corpora whose flip rates differ by at least $10%$, KL assigns the smaller divergence to the one that changes more decisions in $11%$ of cases, total variation in $1%$. Two pre-registered tests mark the limits: on a held-out code corpus the ratio held for all eight models while three predictions about KL each failed for half of them or more, and on three new models with real kernels it stayed in its band for 37 of 38 checkpoints but fell below one on code for two models. In vLLM speculative decoding, total variation measured under teacher forcing predicts greedy draft acceptance with a mean relative error of $1.1$--$2.4%$, without the task-specific calibration that KL needs.
CommentsPreprint
