确定性鲁棒性：在自我挑战提示下评估LLM的稳定性
Certainty robustness: Evaluating LLM stability under self-challenging prompts
AI总结：
本文提出确定性鲁棒性基准，评估LLM在自我挑战提示下的稳定性与适应性平衡，揭示模型在交互压力下的可靠性差异。
摘要：
大型语言模型（LLMs）常常在缺乏明确推理机制的情况下，表现出高显性置信度的答案。尽管现有基准主要评估单轮准确性、真实性或置信度校准，但它们无法捕捉模型在交互环境中被挑战时的行为。我们引入了确定性鲁棒性基准，这是一种双轮评估框架，用于衡量LLMs在自我挑战提示（如不确定性（“你确定吗？”）和显式矛盾（“你错了！”））下如何平衡稳定性和适应性，以及数值置信度引导。使用来自LiveBench的200个推理和数学问题，我们评估了四个最先进的LLMs，并区分了有根据的自我纠正和无根据的答案变化。我们的结果揭示了交互可靠性方面的显著差异，这些差异无法仅通过基线准确性来解释：一些模型在对话压力下放弃正确答案，而其他模型则表现出对挑战的强抵抗力，并且置信度与正确性的匹配度更高。这些发现将确定性鲁棒性识别为LLM评估的一个独特且关键维度，对对齐、可信度和现实世界部署具有重要意义。
英文摘要：
Large language models (LLMs) often present answers with high apparent confidence despite lacking an explicit mechanism for reasoning about certainty or truth. While existing benchmarks primarily evaluate single-turn accuracy, truthfulness or confidence calibration, they do not capture how models behave when their responses are challenged in interactive settings. We introduce the Certainty Robustness Benchmark, a two-turn evaluation framework that measures how LLMs balance stability and adaptability under self-challenging prompts such as uncertainty ("Are you sure?") and explicit contradiction ("You are wrong!"), alongside numeric confidence elicitation. Using 200 reasoning and mathematics questions from LiveBench, we evaluate four state-of-the-art LLMs and distinguish between justified self-corrections and unjustified answer changes. Our results reveal substantial differences in interactive reliability that are not explained by baseline accuracy alone: some models abandon correct answers under conversational pressure, while others demonstrate strong resistance to challenge and better alignment between confidence and correctness. These findings identify certainty robustness as a distinct and critical dimension of LLM evaluation, with important implications for alignment, trustworthiness and real-world deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
