arXiv 2405.125792024-05-23cs.CL

挖掘可解释性与泛化性：基于自我指令的事实核查

Mining the Explainability and Generalization: Fact Verification Based on Self-Instruction

College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与通信电子学院）

Guangyao Lu, Yulin Liu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种基于自我指令的微调方法，结合数据增强和改进的DPO，在保持可解释性的同时提升事实核查的准确性和泛化性，并在FEVEROUS和HOVER数据集上验证了其有效性。

摘要：

基于商业大语言模型（LLM）的事实核查已成为主流。尽管这些方法具有较高的可解释性，但在准确性上不如传统的微调方法，且数据安全也是一个重要问题。在本文中，我们提出了一种基于自我指令的微调方法用于事实核查，以平衡准确性和可解释性。我们的方法包括数据增强和改进的DPO（直接偏好优化）微调。前者首先指导模型基于声明-证据对和标签生成正面和负面的解释，然后根据我们定制的难度标准对数据集进行采样。后者采用我们提出的改进DPO，使用生成的样本对模型进行微调。我们微调了最小规模的LLaMA-7B模型，并在具有挑战性的事实核查数据集FEVEROUS和HOVER上进行了评估，使用了四种微调方法和三种少样本学习方法进行比较。实验表明，我们的方法不仅保持了与传统微调方法相当甚至超越的准确性，而且生成了流畅的解释文本。此外，它还表现出较高的泛化性能。我们的方法是首个将自监督学习用于事实核查的方法，并创新性地在微调大语言模型时结合了对比学习和改进的DPO，如实验所示。

英文摘要：

Fact-checking based on commercial LLMs has become mainstream. Although these methods offer high explainability, it falls short in accuracy compared to traditional fine-tuning approaches, and data security is also a significant concern. In this paper, we propose a self-instruction based fine-tuning approach for fact-checking that balances accuracy and explainability. Our method consists of Data Augmentation and Improved DPO fine-tuning. The former starts by instructing the model to generate both positive and negative explanations based on claim-evidence pairs and labels, then sampling the dataset according to our customized difficulty standards. The latter employs our proposed improved DPO to fine-tune the model using the generated samples. We fine-tune the smallest-scale LLaMA-7B model and evaluate it on the challenging fact-checking datasets FEVEROUS and HOVER, utilizing four fine-tuning methods and three few-shot learning methods for comparison. The experiments demonstrate that our approach not only retains accuracy comparable to, or even surpassing, traditional fine-tuning methods, but also generates fluent explanation text. Moreover, it also exhibit high generalization performance. Our method is the first to leverage self-supervised learning for fact-checking and innovatively combines contrastive learning and improved DPO in fine-tuning LLMs, as shown in the experiments.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。