教大语言模型从自身信号表达知识边界
Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals
AI总结：
本文提出CoKE，利用LLM内部置信度探测其知识边界并据此训练模型表达边界，从而在回答已知问题的同时拒绝未知问题，显著减少幻觉并提升域内外性能。
摘要：
大语言模型（LLM）已经取得了巨大成功，但其偶发的内容捏造（即幻觉）限制了其实际应用。幻觉之所以产生，是因为LLM在知识边界方面训练不足，难以承认自己的无知。我们将LLM无法准确表达其知识边界——即回答自己知道的问题，同时对自己不知道的问题承认无知——称为其局限性。本文旨在教LLM识别并表达其知识边界，从而减少因在不知道时捏造内容而导致的幻觉。我们提出CoKE：给定一组问题，它首先通过内部置信度探测LLM的知识边界，然后利用探测结果引出知识边界的表达。大量实验表明，CoKE有助于LLM表达知识边界，回答已知问题同时拒绝未知问题，显著提升了域内和域外性能。
英文摘要：
Large language models (LLMs) have achieved great success, but their occasional content fabrication, or hallucination, limits their practical application. Hallucination arises because LLMs struggle to admit ignorance due to inadequate training on knowledge boundaries. We call it a limitation of LLMs that they can not accurately express their knowledge boundary, answering questions they know while admitting ignorance to questions they do not know. In this paper, we aim to teach LLMs to recognize and express their knowledge boundary, so they can reduce hallucinations caused by fabricating when they do not know. We propose CoKE, which first probes LLMs' knowledge boundary via internal confidence given a set of questions, and then leverages the probing results to elicit the expression of the knowledge boundary. Extensive experiments show CoKE helps LLMs express knowledge boundaries, answering known questions while declining unknown ones, significantly improving in-domain and out-of-domain performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
