发表机构
Nokia; Tencent Hunyuan; The Hong Kong Polytechnic University（诺基亚; 腾讯混元; 香港理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
AnyJev 通过一次预填充从指令调优语言模型中读取类型化决策，无需训练即可纠正标签和位置偏差，并在多个模型上提升准确率，同时减少预填充次数以加速服务。
AI 中文摘要
类型化决策是在一组固定选项中进行选择，并以概率而非文本形式返回。当前需要类型化决策的系统使用为此目的训练的模型。本报告描述了 AnyJev，它从预训练指令调优语言模型的一次预填充中读取类型化决策。读出操作将答案位置的下一词元分布限制为选项词元。它有两个缺陷：模型对某些标签赋予较高概率，无论输入如何；以及对选项列表中的某些位置赋予较高概率。AnyJev 无需梯度步骤和参数更改即可纠正这两个缺陷：它除以从无标签输入估计的标签先验，并对选项列表的 K 个循环旋转的日志概率进行平均。在两个 20 选项任务上，旋转将顺序翻转率从 0.33 降至 0.14 和从 0.33 降至 0.18，并在两个任务上的 11 个模型中的 11 个上提高了准确率。读取每次旋转需要 K 次预填充。一个针对无标签状态的全旋转决策选择的停止规则减少了这一点。在一个无标签分割上选择阈值并在第二个分割上限制其不一致性，它读取 18 次旋转中的 10.6 次，在四个单元中的两个上验证了 0.008 的界限；如我们的服务运行所做，在一个分割上选择和限制，它读取 7.3 次，并在 vLLM 上每秒服务 2.2 倍多的决策。代码是开源的。
英文摘要
A typed decision is a choice among a fixed set of options, returned as a probability rather than as text. Systems that need typed decisions today use models trained for that purpose. This report describes AnyJev, which reads a typed decision from one prefill of a pretrained instruction-tuned language model. The readout restricts the next-token distribution at the answer position to the option tokens. It has two defects: the model assigns higher probability to some labels whatever the input, and to some positions in the option list. AnyJev corrects both with no gradient steps and no parameter changes: it divides out a label prior estimated from unlabelled inputs, and it averages log-probabilities over the K cyclic rotations of the option list. On two 20-option tasks the rotations lower the order-flip rate from 0.33 to 0.14 and from 0.33 to 0.18, and raise accuracy on 11 of 11 models on both. Reading every rotation requires K prefills. A stopping rule selected against the full-rotation decision on unlabelled states cuts that. Selecting the threshold on one unlabelled split and bounding its disagreement on a second, it reads 10.6 rotations of 18 at a verified 0.008 bound on two of four cells; selected and bounded on one split, as our serving run did, it reads 7.3 and serves 2.2 times as many decisions per second on vLLM. The code is open source.
Comments22 pages, 5 figures. Early report on work in development. Code: https://github.com/nokia-applied-research/AnyJev
