PassiveQA: 一个基于监督微调的三动作框架,用于通过监督微调进行知识校准的问题回答
PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning
AI总结:
本文提出PassiveQA框架,通过监督微调解决信息不全时的问题回答,提升宏F1和退避召回率,减少幻觉。
摘要:
大型语言模型(LLMs)在问答和检索增强生成(RAG)中表现优异,但隐含假设用户查询完全指定且可回答。在现实场景中,查询常不完整、模糊或缺失关键变量,导致模型产生过度自信或幻觉响应。本文研究在信息不全情况下决策感知的查询解决,模型需决定回答、请求澄清或退避。研究表明标准和增强RAG系统无法可靠展示这种知识意识,信息不足时仍默认生成回答。为此,我们提出PassiveQA,一个三动作框架,通过监督微调对齐模型行为与信息充分性。我们的方法整合了结构化信息状态表示、知识图谱基础上下文和一个微调的规划器,显式建模缺失变量和决策推理。在多个QA数据集上的实验表明,微调的规划器在宏F1和退避召回率上有显著提升,同时在计算受限的训练制度下减少幻觉率。这些结果提供了强有力的实证证据,表明知识决策必须在训练期间学习,而非在推理时施加。
英文摘要:
Large Language Models (LLMs) have achieved strong performance in question answering and retrieval-augmented generation (RAG), yet they implicitly assume that user queries are fully specified and answerable. In real-world settings, queries are often incomplete, ambiguous, or missing critical variables, leading models to produce overconfident or hallucinated responses. In this work, we study decision-aware query resolution under incomplete information, where a model must determine whether to Answer, Ask for clarification, or Abstain. We show that standard and enhanced RAG systems do not reliably exhibit such epistemic awareness, defaulting to answer generation even when information is insufficient. To address this, we propose PassiveQA, a three-action framework that aligns model behaviour with information sufficiency through supervised finetuning. Our approach integrates structured information-state representations, knowledge graph-grounded context, and a finetuned planner that explicitly models missing variables and decision reasoning. Experiments across multiple QA datasets show that the finetuned planner achieves significant improvements in macro F1 and abstention recall while reducing hallucination rates, under a compute-constrained training regime. These results provide strong empirical evidence that epistemic decision-making must be learned during training rather than imposed at inference time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。