通过扰动的基于接地感知敏感性检测检索增强生成中的幻觉（GASP）
Sentence-Level Context Sensitivity as a Training-Free Detector of Unsupported Content, Evaluated Against Trained Verifiers
浏览论文内容
中文总结 AI 辅助
研究检索增强生成中幻觉检测问题，提出GASP方法，通过固定答案重新打分，衡量对数似然下降和Jensen-Shannon散度，评估显示其在RAGTruth上效果显著，信号能转移到TofuEval。
中文摘要 AI 辅助
检索增强生成（RAG）减少但未消除幻觉，现有检测器返回单个答案级分数。本文引入GASP，一种跨度级检测器，通过答案句子似然依赖检索证据的强度打分。通过固定答案在不同语境下重新打分，衡量对数似然下降和Jensen-Shannon散度。在三个基准上评估，结果显示其性能优势及适用场景。
英文摘要
Retrieval-augmented generation (RAG) assistants summarize records in clinical and legal work, where one unsupported sentence can mislead a reader. The contrast between an output's likelihood with and without its source is an established faithfulness score for whole summaries and answers, but it has not been measured as a detector of the individual unsupported sentence in multi-passage RAG answers, against trained verifiers, or for its cost. We implement it as a training-free detector that re-scores a fixed answer under the full context, no context, and each chunk removed, and returns the chunk whose removal lowers a sentence's likelihood most as a candidate supporting passage. We evaluate it on RAGTruth, TofuEval, and RAGBench with six scorers and against five verifiers, up to a large language model (LLM) judge, on identical inputs under a source-level split. Scoring per sentence ranks unsupported sentences better than the answer-level form of the same signal on all three benchmarks, by 0.033 to 0.071 in the area under the receiver operating characteristic curve (AUC). On RAGTruth the training-free score reaches an AUC of 0.717 to 0.745 across scorers and 0.773 with a classifier, above entailment and attribution baselines and level with per-chunk fact-checkers, at about one forty-seventh of the LLM judge's compute on a 1.5B scorer, while a full-context fact-checker and the judge are more accurate and are not improved by it. The signal is weakest on short-answer question answering, where the scorer can answer from memory.
发表机构
- Centre for Intelligent Cloud Computing, CoE for Advanced Cloud, Faculty of Information Science and Technology, Multimedia University（智能云计算中心、高级云研究中心、信息科学与技术学院、多媒体大学）
机构由 AI 辅助整理，请以论文原文为准。
