通过保形弃权（不执行）缓解大语言模型幻觉
Mitigating LLM Hallucinations via Conformal Abstention
AI总结：
本文提出一种基于自评估相似性和保形预测的弃权（不执行）方法，用于在通用问答中可靠控制大语言模型的幻觉率，并在长响应数据集上显著降低了保守弃权（不执行）率。
摘要：
我们开发了一个有原则的程序，用于确定大语言模型（LLM）在一般领域中何时应弃权（不执行）回应（例如，通过说“我不知道”），而不是诉诸可能“幻觉”出无意义或不正确的答案。基于早期使用自一致性作为模型置信度更可靠度量的方法，我们提出利用 LLM 自身来对给定查询的每个采样响应之间的相似性进行自我评估。然后，我们进一步利用保形预测技术开发一种弃权（不执行）程序，该程序受益于对幻觉率（错误率）的严格理论保证。在实验中，我们得到的保形弃权（不执行）方法在各种闭卷、开放域生成式问答数据集上可靠地限制了幻觉率，同时在具有长响应的数据集（Temporal Sequences）上与使用对数概率分数来量化不确定性的基线相比，保持了显著更低的保守弃权（不执行）率，并在具有短答案的数据集（TriviaQA）上取得了可比的性能。为了自动评估实验，需要确定在给定问题的情况下两个响应是否等价。按照标准做法，我们使用带阈值的相似性函数来确定两个响应是否匹配，但也提供了一种基于保形预测来校准阈值的方法，并对匹配预测的准确性提供理论保证，这可能具有独立的研究意义。
英文摘要：
We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a non-sensical or incorrect answer. Building on earlier approaches that use self-consistency as a more reliable measure of model confidence, we propose using the LLM itself to self-evaluate the similarity between each of its sampled responses for a given query. We then further leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate). Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets, while also maintaining a significantly less conservative abstention rate on a dataset with long responses (Temporal Sequences) compared to baselines using log-probability scores to quantify uncertainty, while achieveing comparable performance on a dataset with short answers (TriviaQA). To evaluate the experiments automatically, one needs to determine if two responses are equivalent given a question. Following standard practice, we use a thresholded similarity function to determine if two responses match, but also provide a method for calibrating the threshold based on conformal prediction, with theoretical guarantees on the accuracy of the match prediction, which might be of independent interest.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
