PURR：通过去噪语言模型损坏来高效编辑语言模型幻觉
PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions
AI总结：
本文提出PURR模型，通过利用大型语言模型引入噪声并微调紧凑型编辑器进行去噪，实现完全无监督的幻觉修正，在提升归因效果的同时大幅加快执行速度。
摘要：
大型语言模型的卓越能力伴随着一个持续存在的缺陷：生成虚假且无根据的声明，通常被称为“幻觉”。为了解决这一问题，近期的研究引入了编辑和归因语言模型输出的方法，特别是基于提示的编辑。然而，使用大型语言模型进行编辑的推理成本和速度目前成为基于提示方法的瓶颈。这些瓶颈促使了紧凑型编辑器的训练，但由于用于此目的的训练数据稀缺，这极具挑战性。为了克服这些挑战，我们利用大型语言模型的能力向文本中引入损坏（即噪声），随后通过结合相关证据来微调紧凑型编辑器以对损坏进行去噪。我们的方法完全无监督，并为我们提供了用于在任何领域进行训练的模拟幻觉。我们的微型无监督研究与修订模型（PURR）不仅改进了基于微调和提示的现有编辑方法的归因效果，而且在执行速度上实现了数量级的提升。
英文摘要：
The remarkable capabilities of large language models have been accompanied by a persistent drawback: the generation of false and unsubstantiated claims commonly known as "hallucinations". To combat this issue, recent research has introduced approaches that involve editing and attributing the outputs of language models, particularly through prompt-based editing. However, the inference cost and speed of using large language models for editing currently bottleneck prompt-based methods. These bottlenecks motivate the training of compact editors, which is challenging due to the scarcity of training data for this purpose. To overcome these challenges, we exploit the power of large language models to introduce corruptions (i.e., noise) into text and subsequently fine-tune compact editors to denoise the corruptions by incorporating relevant evidence. Our methodology is entirely unsupervised and provides us with faux hallucinations for training in any domain. Our Petite Unsupervised Research and Revision model, PURR, not only improves attribution over existing editing methods based on fine-tuning and prompting, but also achieves faster execution times by orders of magnitude.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
