超越置信度：面向LLM推理的稳定性感知测试时自适应

中文摘要 AI 辅助

测试时自适应已成为一种轻量级替代方案，用于替代昂贵的后训练，以提升大型语言模型（LLMs）在下游任务上的推理能力。预测熵为这种自适应提供了模型衍生的信号，引导模型在无需外部验证器或奖励模型的情况下走向更高置信度的推理状态。然而，更高的置信度并不一定意味着正确性，因为LLMs可能沿着错误的推理轨迹保持高度自信。我们观察到，当置信度在局部扰动下保持稳定时，高置信度的推理更可能是正确的。基于这一观察，我们提出了通过稳定性感知置信度优化的测试时自适应（TASCO），这是一个将局部稳定性纳入基于置信度的测试时自适应并保持LLM冻结的框架。TASCO通过优化轻量级任务级前缀来实施局部稳定性，采用两种替代扰动策略：随机扰动促进由附近扰动前缀引发的轨迹间的分布稳定性，而锐度感知扰动则针对最坏情况的局部敏感性。实验表明，TASCO在多种LLMs和推理基准上提高了推理准确性和令牌效率，同时行为分析显示，它在局部扰动下保持稳定的置信度，而不会过早地集中模型的预测分布。

英文摘要

Test-time adaptation has emerged as a lightweight alternative to costly post-training for improving the reasoning capabilities of Large Language Models (LLMs) on downstream tasks. Predictive entropy provides a model-derived signal for such adaptation, guiding models toward higher-confidence reasoning states without external verifiers or reward models. However, higher confidence does not necessarily imply correctness, as LLMs may remain highly confident along incorrect reasoning trajectories. We observe that high-confidence reasoning is more likely to be correct when confidence remains stable under local perturbations. Based on this observation, we propose Test-Time Adaptation via Stability-Aware Confidence Optimization (TASCO), a framework that incorporates local stability into confidence-based test-time adaptation while keeping the LLM frozen. TASCO operationalizes local stability by optimizing a lightweight task-level prefix under two alternative perturbation strategies: Random Perturbation promotes distributional stability across trajectories induced by nearby perturbed prefixes, whereas Sharpness-Aware Perturbation targets worst-case local sensitivity. Experiments demonstrate that TASCO improves reasoning accuracy and token efficiency across diverse LLMs and reasoning benchmarks, while behavioral analyses show that it maintains stable confidence under local perturbations without prematurely concentrating the model's predictive distribution.

发表机构

• Chongqing University（重庆大学）
• Griffith University（格里菲斯大学）

机构由 AI 辅助整理，请以论文原文为准。