答案在哪里？探究语言模型知识提取中的位置偏差
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction
AI总结：
针对大型语言模型微调后存在的“困惑度诅咒”问题，研究发现其知识提取存在位置偏差，即易提取文档首句信息、难提取中间或结尾信息，通过发布相关数据集及验证去噪自回归损失等正则化方法的改善作用，为改进知识提取及讨论RAG与微调权衡提供新依据。
摘要：
大型语言模型（LLMs）需要通过使用新文档微调来保持最新状态或适应新领域。关键在于以可通过查询提示提取的方式记忆最新信息。然而，LLMs存在一种称为“困惑度诅咒”的现象：尽管微调期间最小化了文档困惑度，LLMs仍难以通过提示句提取信息。在这项新知识获取与提取研究中，我们发现一个有趣事实：LLMs能准确回答关于文档第一句的问题，但难以提取微调所用文档中间或结尾描述的信息。研究表明，自回归训练导致该问题——每个token依赖所有先前token生成，阻碍模型通过问题提示回忆训练文档信息。为深入研究，我们发布合成与真实数据集，可评估问题答案在文档中位置相关的问答（QA）性能。调查显示，即使大型模型也受困惑度诅咒影响，但去噪自回归损失等正则化方法可增强从不同位置的信息提取。这些发现将成为（i）改进LLMs知识提取的关键，（ii）讨论RAG与微调在LLMs适应新领域中的权衡的新要素。
英文摘要：
Large language models require updates to remain up-to-date or adapt to new domains by fine-tuning them with new documents. One key is memorizing the latest information in a way that the memorized information is extractable with a query prompt. However, LLMs suffer from a phenomenon called perplexity curse; despite minimizing document perplexity during fine-tuning, LLMs struggle to extract information through a prompt sentence. In this new knowledge acquisition and extraction, we find a very intriguing fact that LLMs can accurately answer questions about the first sentence, but they struggle to extract information described in the middle or end of the documents used for fine-tuning. Our study suggests that the auto-regressive training causes this issue; each token is prompted by reliance on all previous tokens, which hinders the model from recalling information from training documents by question prompts. To conduct the in-depth study, we publish both synthetic and real datasets, enabling the evaluation of the QA performance w.r.t. the position of the corresponding answer in a document. Our investigation shows that even a large model suffers from the perplexity curse, but regularization such as denoising auto-regressive loss can enhance the information extraction from diverse positions. These findings will be (i) a key to improving knowledge extraction from LLMs and (ii) new elements to discuss the trade-off between RAG and fine-tuning in adapting LLMs to a new domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
