将大语言模型的领域自适应重构为自适应-检索-修订：以中国法律领域为例
Reformulating Domain Adaptation of Large Language Models as Adapt-Retrieve-Revise: A Case Study on Chinese Legal Domain
AI总结：
本文提出一种 adapt-retrieve-revised 框架，通过小模型生成草稿、检索外部证据并由 GPT-4 修订，有效解决大模型在特定领域的幻觉问题，在中国法律任务上显著提升准确率。
摘要：
尽管像 GPT-4 这样的大语言模型（LLMs）最近在通用领域任务中展现了惊人的零样本能力，但它们在特定领域（如中国法律）中经常生成带有幻觉的内容，阻碍了其在这些领域的应用。这通常是由于缺乏涵盖此类特定领域的训练数据，导致 GPT-4 无法获取领域内知识。一个紧迫的挑战是，在此类规模的大语言模型上继续使用领域内数据进行训练是不可行的。本文通过将生成过程重构为自适应-检索-修订过程，引入了一个简单有效的 GPT-4 领域自适应框架。第一步是通过在领域内数据上继续学习，将一个可负担的 7B LLM 自适应到目标领域。在解决任务时，我们利用自适应后的 LLM 根据任务查询生成草稿答案。然后，草稿答案将用于从外部领域内知识库中检索支持证据候选。最后，草稿答案和检索到的证据被拼接成一个完整的提示，让 GPT-4 评估证据并修订草稿答案以生成最终答案。我们的提案结合了自适应较小 7B 模型的效率优势与 GPT-4 的证据评估能力，并有效防止了 GPT-4 生成幻觉内容。在四个中国法律任务的零样本设置中，与 GPT-4 直接生成相比，我们的方法将准确率提高了 33.3%。与两个更强的基于检索的基线相比，我们的方法分别优于它们 15.4% 和 23.9%。我们的代码将发布。
英文摘要：
While large language models (LLMs) like GPT-4 have recently demonstrated astonishing zero-shot capabilities in general domain tasks, they often generate content with hallucinations in specific domains such as Chinese law, hindering their application in these areas. This is typically due to the absence of training data that encompasses such a specific domain, preventing GPT-4 from acquiring in-domain knowledge. A pressing challenge is that it's not plausible to continue training LLMs of such scale on in-domain data. This paper introduces a simple and effective domain adaptation framework for GPT-4 by reformulating generation as an \textbf{adapt-retrieve-revise} process. The initial step is to \textbf{adapt} an affordable 7B LLM to the target domain by continuing learning on in-domain data. When solving a task, we leverage the adapted LLM to generate a draft answer given a task query. Then, the draft answer will be used to \textbf{retrieve} supporting evidence candidates from an external in-domain knowledge base. Finally, the draft answer and retrieved evidence are concatenated into a whole prompt to let GPT-4 assess the evidence and \textbf{revise} the draft answer to generate the final answer. Our proposal combines the advantages of the efficiency of adapting a smaller 7B model with the evidence-assessing capability of GPT-4 and effectively prevents GPT-4 from generating hallucinatory content. In the zero-shot setting of four Chinese legal tasks, our method improves accuracy by 33.3\% compared to the direct generation by GPT-4. When compared to two stronger retrieval-based baselines, our method outperforms them by 15.4\% and 23.9\%. Our code will be released
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
