通过身份桥打破自回归语言模型中的逆转诅咒
Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge
AI总结：
提出一种名为“身份桥”的简单数据正则化方法（形式为“A→A”），通过理论分析和实验证明该方法能有效缓解自回归语言模型中的逆转诅咒，使模型从事实记忆转向规则学习。
摘要：
自回归大型语言模型（LLMs）在许多复杂任务中取得了显著成功，但在非常简单的逻辑推理中仍可能失败，例如“逆转诅咒”——当模型在形如“$A \rightarrow B$”（例如，爱丽丝的丈夫是鲍勃）的前向知识数据上训练时，在测试时无法推断出逆转知识“$B \leftarrow A$”（例如，鲍勃的妻子是爱丽丝）。大量先前的研究表明，这种失败是自回归因果LLMs固有的根本限制，表明这些模型倾向于记忆事实层面的知识，而不是捕捉更高级别的规则。在本文中，我们通过展示这种看似根本的限制可以通过略微调整训练数据，使用一种简单的正则化数据配方（称为“身份桥”，形式为“$A \to A$”，例如，爱丽丝的名字是爱丽丝）来缓解，从而挑战了这一观点。理论上，我们证明在这种配方下，即使是一层Transformer也可以通过分析梯度下降的隐式偏差来打破逆转诅咒。实验上，我们展示了一个10亿参数的预训练语言模型，在使用所提出的数据配方进行微调后，在逆转任务上达到了50%的成功率，而仅在前向知识数据上训练时成功率接近零。我们的工作为逆转诅咒提供了新颖的理论基础，并为鼓励LLMs从数据中学习更高级别的规则提供了一条原则性、低成本的路径。
英文摘要：
Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.g., Alice's husband is Bob), the model is unable to deduce the reversal knowledge "$B \leftarrow A$" (e.g., Bob's wife is Alice) during test. Extensive prior research suggests that this failure is an inherent, fundamental limit of autoregressive causal LLMs, indicating that these models tend to memorize factual-level knowledge rather than capture higher-level rules. In this paper, we challenge this view by showing that this seemingly fundamental limit can be mitigated by slightly tweaking the training data with a simple regularization data recipe called the Identity Bridge of the form "$A \to A$" (e.g., The name of Alice is Alice). Theoretically, we prove that under this recipe, even a one-layer transformer can break the reversal curse by analyzing the implicit bias of gradient descent. Empirically, we show that a 1B pretrained language model finetuned with the proposed data recipe achieves a 50% success rate on reversal tasks, in stark contrast to a near-zero success rate when trained solely on forward-knowledge data. Our work provides a novel theoretical foundation for the reversal curse and offers a principled, low-cost path to encouraging LLMs to learn higher-level rules from data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
