深入探究反转诅咒：大语言模型能泛化到多远？
Delving into the Reversal Curse: How Far Can Large Language Models Generalize?
AI总结：
本文系统研究了大语言模型中的“反转诅咒”现象，揭示其泛化能力依赖于上下文呈现方式和训练文档结构，并验证了模型在事实回忆中的固有偏差及其难以仅靠训练消除的特性。
摘要：
尽管大语言模型（LLMs）展现出前所未有的能力，但它们在面对看似简单的任务时也表现出某些固有的局限性。一个典型的例子是最近备受争议的“反转诅咒”，当模型在“A 是 B”这一事实上进行训练后，难以将这一知识泛化为推断“B 是 A”时，该现象就会出现。在本文中，我们考察了反转诅咒在多种任务中的表现，并深入探究了 LLMs 的泛化能力和问题解决机制。这项研究带来了一系列重要发现：（1）当 A 和 B 都像多项选择题那样出现在上下文中时，LLMs 能够泛化到“B 是 A”。（2）这种泛化能力与训练文档中“A 是 B”这一事实的结构高度相关。例如，这种泛化仅适用于以“[姓名] 是 [描述]”结构组织的传记，而不适用于“[描述] 是 [姓名]”。（3）我们提出并验证了如下假设：LLMs 在知识应用过程中进行事实回忆时存在固有偏差，这解释并强调了文档结构对成功学习的重要性。（4）这种偏差对 LLMs 下游性能的负面影响很难仅通过训练来缓解。这些发现为通过 LLMs 的内在机制解释其泛化能力提供了新颖视角，并为开发更有效的学习方法提供了启示。我们的代码和数据可在 https://github.com/alibaba/thinking_bias.git 获取。
英文摘要：
While large language models (LLMs) showcase unprecedented capabilities, they also exhibit certain inherent limitations when facing seemingly trivial tasks. A prime example is the recently debated "reversal curse", which surfaces when models, having been trained on the fact "A is B", struggle to generalize this knowledge to infer that "B is A". In this paper, we examine the manifestation of the reversal curse across various tasks and delve into both the generalization abilities and the problem-solving mechanisms of LLMs. This investigation leads to a series of significant insights: (1) LLMs are able to generalize to "B is A" when both A and B are presented in the context as in the case of a multiple-choice question. (2) This generalization ability is highly correlated to the structure of the fact "A is B" in the training documents. For example, this generalization only applies to biographies structured in "[Name] is [Description]" but not to "[Description] is [Name]". (3) We propose and verify the hypothesis that LLMs possess an inherent bias in fact recalling during knowledge application, which explains and underscores the importance of the document structure to successful learning. (4) The negative impact of this bias on the downstream performance of LLMs can hardly be mitigated through training alone. These findings offer a novel perspective on interpreting LLMs' generalization through their intrinsic mechanisms and provide insights for developing more effective learning methods. Our code and data are available at https://github.com/alibaba/thinking_bias.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
