两次调用、两次矩与重复LLM推理的投票准确性曲线
Two Calls, Two Moments, and the Vote-Accuracy Curve of Repeated LLM Inference
AI总结：
研究重复LLM推理的二元正确性层，在条件独立同分布调用下，通过两次调用确定第二矩和相同示例正确性相关性，从而为固定多数投票预算提供分布无关的两调用区间。
摘要：
重复采样是测试时间计算的标准消耗方式，但其效益由正确性在示例中的潜在分布控制，而非单一调用准确性。我们研究了在条件i.i.d.调用下重复LLM推理的二元正确性层。一个标记调用确定了潜藏成功概率的均值；两个标记调用确定了其二阶矩，从而分离稳定误差与可恢复的调用级随机性。从这两个矩值，每个固定多数投票预算都有一个分布无关的两调用区间。关键的技术减少是无限维矩问题有三原子极值器和二次对偶证书，因此界限是精确的而非离散或参数化的。第一个有用的预算，三次投票，有闭合形式，宽度最大为1/8，并有一个认证改进标准。无限投票端点是多数投票随调用数趋于无穷时的极限；它也严格有界，但仍然对阈值敏感，因为它依赖于潜藏质量在q=1/2附近的质量。我们添加了最大熵和潜藏难度高斯-概率点补全，并在QNLI和QQP上的LLM调用实验显示，经验三次和五次投票准确性包含在投影的两调用区域内，而温度变化和随机模型混合可以创造投票增益，这些增益不受单次调用准确性排序的影响。
英文摘要：
Repeated sampling can improve LLM accuracy, and quantifying the gains from additional calls is essential for allocating test-time compute. We study binary decisions, a fundamental setting where repeated answers to the same question are aggregated by majority vote. We show that two independently sampled responses per example in a validation set with known answers constrain the latent distribution of example-level success probabilities to a class consistent with the paired outcomes. Optimizing over this class yields sharp accuracy and gain bounds at every finite voting budget. A shared large-sample confidence region accounts for validation uncertainty. We also obtain sharp infinite-vote bounds and moment-matched forecasts of the vote-accuracy curve. On QNLI and QQP, our method successfully distinguishes settings with voting gains above a target margin from those with little room for improvement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
