用于诊断推理模型中未知未知的结构化无知证书的校准
Calibration of Structured Ignorance Certificates for Diagnosing Unknown Unknowns in Reasoning Models
AI总结:
提出结构化无知证书(SICs)输出格式,通过GRPO微调14B模型,使模型在无法回答时明确承认知识缺失并生成检索查询,在未知未知问题上实现99.46%的JSON有效性和0.967的证书特异性分数。
摘要:
大型语言模型经常以特征性方式失败:对于超出其知识边界的问题,它们不是承认无知,而是生成流畅但错误的答案。我们引入了\textbf{结构化无知证书}(SICs),这是一种JSON格式的输出模式,要求模型明确命名缺失的领域交叉点,列举所需概念,并提出一个富有成效的检索查询,而不是凭空捏造答案。为了训练模型生成高质量的SICs,我们构建了一个包含7,347个样本的\emph{未知-未知}(UU)数据集,通过提示Qwen3-14B将来自七个领域(物理、生物、工程、计算机科学、经济、医学、法律)的问题拼接成新颖的跨领域查询,这些查询是任何单一领域专家都无法回答的。我们使用组相对策略优化(GRPO)微调了一个14B参数的模型,采用结合检索效用、概念特异性和输出格式有效性的复合奖励。在模型响应上训练的释义散度探测器证实,SIC调优的输出系统地表现出更高的未知-未知概率分数。在735个保留的UU问题上的评估实现了99.46%的JSON有效性率、0.967的平均证书特异性分数,以及在基于检索的生成上相比基础模型3.6%的ROUGE-L改进——这表明显式的认知结构化是一种可学习且可衡量的能力。
英文摘要:
Large language models frequently fail in a characteristic way: rather than acknowledging ignorance, they produce fluent but incorrect answers to questions that lie beyond their knowledge boundaries. We introduce \textbf{Structured Ignorance Certificates} (SICs), a JSON-formatted output schema that demands a model explicitly name the missing domain intersection, enumerate required concepts, and propose a productive retrieval query rather than hallucinating an answer. To train models to produce high-quality SICs we construct a 7,347-sample \emph{Unknown-Unknown} (UU) dataset by prompting Qwen3-14B to stitch together questions from seven domains (physics, biology, engineering, CS, economics, medical, legal) into novel cross-domain queries that no single-domain expert could answer. We fine-tune a 14B-parameter model with Group Relative Policy Optimization (GRPO) using a composite reward that combines retrieval utility, concept specificity, and output-format validity. A paraphrase-divergence probe trained on model responses confirms that SIC-tuned outputs systematically exhibit higher unknown-unknown probability scores. Evaluation on 735 held-out UU questions achieves a 99.46\% JSON validity rate, a mean Certificate Specificity Score of 0.967, and a 3.6\% ROUGE-L improvement over the base model on retrieval-grounded generation -- demonstrating that explicit epistemic structuring is a learnable and measurable capability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。