大型语言模型应显式表达不确定性
LLMs Should Express Uncertainty Explicitly
AI总结：
本文研究通过训练使模型在回答时显式表达不确定性，以减少过度自信的错误，提升回答质量，并通过检索增强生成技术改进最终响应。
摘要：
大型语言模型（LLMs）常常产生自信但错误的答案，这可能导致现实应用中的风险失败。我们研究是否可以通过训练使模型在回答时显式表达不确定性：当模型不确定时，能否在自身响应中信号这一点？核心设计问题是该信号应何时暴露——在推理过程中、在答案仍形成时，还是在答案生成后。我们研究了两种方法：对于推理结束时的自我评估，我们训练模型在回答中表达置信度分数，以实现正确答案的高置信度和错误答案的低置信度。对于推理过程中的自我评估，我们训练模型在当前推理状态不可靠时发射<uncertain>标记。在事实推理任务中，两种方法均显著减少过度自信的错误并提高回答质量，且均可作为检索增强生成（RAG）的触发器以改善最终响应。我们进一步分析了其内部机制：推理结束时的显式置信度增强了预训练模型中已存在的置信度相关结构，而推理过程中的<uncertain>发射教会模型标记高风险的推理步骤，参数变化集中在模型的后期层中。
英文摘要：
Large language models (LLMs) often produce confident yet incorrect answers, which can lead to risky failures in real-world applications. We study whether post-training can make a model's self-assessment explicit: when the model is uncertain, can it be trained to signal so within its own response? A central design question is where in the response this signal should be exposed -- during reasoning, while the answer is still being formed, or at the end, once the answer has been produced. We study both. For end-of-reasoning self-assessment, we train the model to verbalize a confidence score for its response, with the aim of high confidence on correct answers and low confidence on incorrect ones. For during-reasoning self-assessment, we train the model to emit the marker <uncertain> whenever its current reasoning state appears unreliable. Across factual reasoning tasks, both forms sharply reduce overconfident errors while improving answer quality, and both can be used as triggers for retrieval augmented generation (RAG) to improve the final response. We further analyze their internal mechanisms: end-of-reasoning verbalized confidence sharpens a confidence-related structure already present in the pretrained model, whereas during-reasoning <uncertain> emission teaches the model to mark high-risk reasoning steps, with parameter changes concentrated in the model's late layers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
