应对不确定性：优化API依赖以减少闭卷问答中的幻觉
Navigating Uncertainty: Optimizing API Dependency for Hallucination Reduction in Closed-Book Question Answering
AI总结：
提出一种能自我评估是否需调用外部API的LLM，通过幻觉掩蔽机制和参数高效微调，在闭卷问答中有效降低幻觉并减少API调用成本。
摘要：
尽管大型语言模型（LLM）能够积累和恢复知识，但它们仍容易产生幻觉。特别是在面对事实性问题时，LLM不能仅依靠存储在参数中的知识来保证答案的真实性和正确性。为这些模型增强在外部信息源（如网络）上搜索的能力，是使知识落地以检索信息的一种有前景的方法。然而，在大量文档集合中进行搜索会引入额外的计算/时间成本。最优行为是仅在LLM对答案不确定时才查询外部资源。在本文中，我们提出了一种新的LLM，能够自我评估它是能够直接回答还是需要请求外部工具。我们研究了一种监督方法，引入了幻觉掩蔽机制，其中标签是使用闭卷问答任务生成的。此外，我们提出利用参数高效微调技术，在少量数据上训练我们的模型。我们的模型对78.2%的已知查询直接提供答案，并对77.2%的未知查询选择进行搜索。这导致API仅在62%的时间被使用。
英文摘要：
While Large Language Models (LLM) are able to accumulate and restore knowledge, they are still prone to hallucination. Especially when faced with factual questions, LLM cannot only rely on knowledge stored in parameters to guarantee truthful and correct answers. Augmenting these models with the ability to search on external information sources, such as the web, is a promising approach to ground knowledge to retrieve information. However, searching in a large collection of documents introduces additional computational/time costs. An optimal behavior would be to query external resources only when the LLM is not confident about answers. In this paper, we propose a new LLM able to self-estimate if it is able to answer directly or needs to request an external tool. We investigate a supervised approach by introducing a hallucination masking mechanism in which labels are generated using a close book question-answering task. In addition, we propose to leverage parameter-efficient fine-tuning techniques to train our model on a small amount of data. Our model directly provides answers for $78.2\%$ of the known queries and opts to search for $77.2\%$ of the unknown ones. This results in the API being utilized only $62\%$ of the time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
