MiRD:通过误覆盖风险分解实现开放式问答的可靠集值预测
MiRD: Reliable Set-Valued Prediction for Open-Ended Question Answering via Miscoverage Risk Decomposition
AI总结:
提出MiRD两阶段框架,通过将整体误覆盖分解为采样失败和条件选择失败,在开放式问答中实现可靠的集值预测,控制采样风险和条件选择风险,并产生更紧的边界和更自适应的预测集。
摘要:
可靠的集值预测为缓解开放式问答中的幻觉提供了一种原则性方法,但现有的共形方法通常依赖于一个脆弱的假设:有限采样必须已经产生至少一个可接受的候选,或者违反此条件的校准示例被丢弃。在本文中,我们介绍了MiRD,一个两阶段框架,将整体误覆盖分解为采样失败和条件选择失败。在第一阶段,MiRD在固定预算下,对有限采样不产生可接受答案的概率建立了一个期望水平的边际上界。在第二阶段,基于采样成功,MiRD使用在整个校准集上定义的与接受性相关的非一致性分数来校准共形选择阈值,从而保持校准集的完整性。在三个开放式问答数据集和八个模型上,MiRD控制了采样风险、条件选择风险和整体误覆盖,同时产生了比PAC风格替代方案更紧的第一阶段边界,以及比仅成功校准更自适应的预测集。
英文摘要:
Reliable set-valued prediction provides a principled way to mitigate hallucinations in open-ended question answering (QA), yet existing conformal approaches typically rely on a fragile premise: finite sampling must already produce at least one admissible candidate, or calibration examples violating this condition are discarded. In this paper, we introduce MiRD, a two-stage framework that decomposes overall miscoverage into sampling failure and conditional selection failure. In Stage I, MiRD establishes an expectation-level marginal upper bound on the probability that finite sampling produces no admissible answer under a fixed budget. In Stage II, conditioned on sampling success, MiRD calibrates a conformal selection threshold using admission-correlated nonconformity scores defined over the full calibration set, thereby preserving calibration-set integrity. Across three open-ended QA datasets and eight models, MiRD controls sampling risk, conditional selection risk, and overall miscoverage, while yielding tighter first-stage bounds than PAC-style alternatives and more adaptive prediction sets than successful-only calibration.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。