语言模型的演绎闭包训练：面向连贯性、准确性与可更新性
Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability
AI总结：
提出演绎闭包训练（DCT）方法，利用语言模型自身识别生成文本的蕴含与矛盾，通过自监督微调提升事实性、连贯性与可更新性，在多个数据集上显著提升验证与生成准确率。
摘要：
尽管语言模型有时能够生成事实正确的文本，并估计单个陈述的真值，但这些通常并不反映一个全局连贯、可操控的世界模型。因此，当前的语言模型也会生成错误或无意义的内容，且难以编辑和更新。我们提出了一种名为演绎闭包训练（DCT）的方法，该方法利用语言模型自身来识别其生成文本的蕴含关系（及内部矛盾），从而产生一种高效的自监督流程以提升语言模型的事实性。给定一组种子文档，DCT 提示语言模型生成由这些文档蕴含的额外文本，全局推理这些生成文本的正确性，最后对推断为正确的文本进行微调。若种子文档来自可信源，DCT 提供了一种用于监督式模型更新的工具；若种子文档从模型自身采样，DCT 则实现了完全无监督的微调，以提升连贯性和准确性。在 CREAK、MQUaKE 和 Reversal Curse 数据集上，监督式 DCT 将语言模型的事实验证和文本生成准确率提升了 3-26%；在 CREAK 上，完全无监督的 DCT 将验证准确率提升了 12%。这些结果表明，语言模型在推理期间的推理能力可在训练期间被利用以提升其可靠性。
英文摘要：
While language models (LMs) can sometimes generate factually correct text and estimate truth values of individual claims, these generally do not reflect a globally coherent, manipulable model of the world. As a consequence, current LMs also generate incorrect or nonsensical content, and are difficult to edit and bring up to date. We present a method called Deductive Closure Training (DCT) that uses LMs themselves to identify implications of (and contradictions within) the text that they generate, yielding an efficient self-supervised procedure for improving LM factuality. Given a collection of seed documents, DCT prompts LMs to generate additional text implied by these documents, reason globally about the correctness of this generated text, and finally fine-tune on text inferred to be correct. Given seed documents from a trusted source, DCT provides a tool for supervised model updating; if seed documents are sampled from the LM itself, DCT enables fully unsupervised fine-tuning for improved coherence and accuracy. Across the CREAK, MQUaKE, and Reversal Curse datasets, supervised DCT improves LM fact verification and text generation accuracy by 3-26%; on CREAK fully unsupervised DCT improves verification accuracy by 12%. These results show that LMs' reasoning capabilities during inference can be leveraged during training to improve their reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
