大型语言模型何时应弃权（不执行）？自我提问链用于选择性风险控制

中文摘要 AI 辅助

大型语言模型在事实支撑薄弱时仍能生成流畅的答案。本文提出自我提问链（Chain-of-Self-Questioning, CoSQ），一个仅基于提示的框架，使答案承诺取决于对回答问题所需信息的显式评估。我们在包含817个条目的TruthfulQA多项选择验证集上，使用十一个开源及托管模型家族，在十七种条件下评估了三种CoSQ变体。在最终的平衡选项协议中，Grounded-CoSQ在τ=0.90时将无条件错误承诺率的平均值从思维链提示下的13.1%降至8.9%，相对降低32.1%，同时将回答准确率从86.9%提升至89.7%，并回答了87.6%的问题。这两项改进在全部十一个模型及所有评估阈值下均成立。Critical-CoSQ和Adaptive-CoSQ提供了相邻的工作点，覆盖率分别为88.6%和86.5%，同时仍比基线更可靠。一项次要的自然问题短答案评估提供了收敛的开放式证据。这些发现表明，当无支撑的承诺比转介或审查代价更高时，自我评估可以支持显式、可调的作答或弃权（不执行）决策。

英文摘要

Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. We evaluate three CoSQ variants under seventeen conditions on the 817-item TruthfulQA multiple-choice validation set using eleven open-weight and hosted model families. In the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9%, a 32.1% relative reduction, while increasing answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements hold for all eleven models and at every evaluated threshold. Critical-CoSQ and Adaptive-CoSQ provide neighboring operating points with 88.6% and 86.5% coverage, respectively, while remaining more reliable than the baseline. A secondary Natural Questions Short-Answer evaluation provides convergent open-form evidence. These findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when an unsupported commitment is more costly than referral or review.

发表机构

• Tarsus University（塔尔苏斯大学）

机构由 AI 辅助整理，请以论文原文为准。