规模扩展、检索增强与形式对语言模型事实一致性的影响
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
AI总结：
本文研究LLM对语义等价问题回答不一致的问题，评估扩大规模与检索增强两种策略，发现二者均有效且检索增强效率更高，并分析了Atlas组件及句法形式等因素。
摘要：
大型语言模型（LLM）为事实知识提供了自然的交互接口，但其效用受到一种倾向的限制：它们会对语义等价的问题给出不一致的答案。例如，同一模型可能既预测“Anne Redpass在爱丁堡去世。”又预测“Anne Redpath的生命在伦敦终结。”在本研究中，我们识别了导致不一致的潜在原因，并评估了两种缓解策略的有效性：扩大模型规模，以及利用检索语料库增强语言模型。我们在LLaMA和Atlas模型上的结果表明，两种策略都能降低不一致性，而检索增强的效率显著更高。我们进一步考察并分解了Atlas不同组件对一致性的贡献。对于所有受评估的语言模型，我们发现句法形式及其他评估任务伪影都会影响一致性。综合而言，我们的结果有助于更好地理解影响语言模型事实一致性的因素。
英文摘要：
Large Language Models (LLMs) make natural interfaces to factual knowledge, but their usefulness is limited by their tendency to deliver inconsistent answers to semantically equivalent questions. For example, a model might predict both "Anne Redpath passed away in Edinburgh." and "Anne Redpath's life ended in London." In this work, we identify potential causes of inconsistency and evaluate the effectiveness of two mitigation strategies: up-scaling and augmenting the LM with a retrieval corpus. Our results on the LLaMA and Atlas models show that both strategies reduce inconsistency while retrieval augmentation is considerably more efficient. We further consider and disentangle the consistency contributions of different components of Atlas. For all LMs evaluated we find that syntactical form and other evaluation task artifacts impact consistency. Taken together, our results provide a better understanding of the factors affecting the factual consistency of language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
