突破预采样障碍：基于激活的难度感知自一致性
Breaking the Pre-Sampling Barrier: Activation-Informed Difficulty-Aware Self-Consistency
AI总结：
ACTSC通过利用神经元激活信号构建轻量级难度估计探针，无需额外调用或预采样，有效降低推理成本并保持准确性。
摘要：
自一致性（SC）是一种有效的解码策略，通过生成多个思维链推理路径并通过多数投票选择最终答案，从而提高大型语言模型（LLMs）的推理性能。然而，它面临显著的推理成本，因为需要大量的样本。为缓解这一问题，提出了难度自适应自一致性（DSC），通过根据问题难度调整样本数量来减少简单问题的不必要的令牌使用。然而，DSC需要额外的模型调用和预采样来估计难度，且在应用到每个数据集时都会重复此过程，导致显著的计算开销。在本工作中，我们提出了一种基于激活的难度感知自一致性（ACTSC），以解决这些限制。ACTSC利用反馈网络神经元激活中反映的内部难度信号，构建一个轻量级的难度估计探针，无需任何额外的令牌生成或模型调用。该探针动态调整SC的样本数量，并可应用于新数据集，而无需预采样进行难度估计。为了验证其有效性，我们在五个基准上进行了实验。实验结果表明，ACTSC在保持准确性的同时有效降低了推理成本，相对于现有方法。
英文摘要：
Self-Consistency (SC) is an effective decoding strategy that improves the reasoning performance of Large Language Models (LLMs) by generating multiple chain-of-thought reasoning paths and selecting the final answer via majority voting. However, it suffers from substantial inference costs because it requires a large number of samples. To mitigate this issue, Difficulty-Adaptive Self-Consistency (DSC) was proposed to reduce unnecessary token usage for easy problems by adjusting the number of samples according to problem difficulty. However, DSC requires additional model calls and pre-sampling to estimate difficulty, and this process is repeated when applying to each dataset, leading to significant computational overhead. In this work, we propose Activation-Informed Difficulty-Aware Self-Consistency (ACTSC) to address these limitations. ACTSC leverages internal difficulty signals reflected in the feed-forward network neuron activations to construct a lightweight difficulty estimation probe, without any additional token generation or model calls. The probe dynamically adjusts the number of samples for SC and can be applied to new datasets without requiring pre-sampling for difficulty estimation. To validate its effectiveness, we conduct experiments on five benchmarks. Experimental results show that ACTSC effectively reduces inference costs while maintaining accuracy relative to existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
