逆转诅咒：基于“A是B”训练的LLM无法学会“B是A”
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
AI总结：
该研究揭示自回归LLM在训练“A是B”后无法泛化到“B是A”，即逆转诅咒，并通过微调GPT-3和Llama-1及评估GPT-4证实其稳健性。
摘要：
我们揭示了自回归大型语言模型（LLMs）中一个令人惊讶的泛化失败。如果模型在形如“A是B”的句子上训练，它不会自动泛化到反向“B是A”。这就是逆转诅咒。例如，如果模型在“瓦莲京娜·捷列什科娃是第一位进入太空的女性”上训练，它不会自动回答“谁是第一位进入太空的女性？”的问题。此外，正确答案（“瓦莲京娜·捷列什科娃”）的可能性不会高于随机名字。因此，模型不会泛化其训练集中的普遍模式：如果“A是B”出现，则“B是A”更可能出现。然而，值得注意的是，如果“A是B”在上下文中出现，模型可以推断出反向关系。我们通过在虚构陈述（如“乌利亚·霍桑是《深渊旋律》的作曲家”）上微调GPT-3和Llama-1，并展示它们无法正确回答“谁创作了《深渊旋律》？”来为逆转诅咒提供证据。逆转诅咒在模型大小和模型家族中都是稳健的，并且数据增强不能缓解它。我们还评估了ChatGPT（GPT-3.5和GPT-4）关于真实世界名人的问题，例如“汤姆·克鲁斯的母亲是谁？[答：玛丽·李·菲佛]”和反向“玛丽·李·菲佛的儿子是谁？”。GPT-4正确回答前者类型的问题79%的时间，而后者只有33%。代码可在：https://github.com/lukasberglund/reversal_curse 获取。
英文摘要：
We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Valentina Tereshkova was the first woman to travel to space", it will not automatically be able to answer the question, "Who was the first woman to travel to space?". Moreover, the likelihood of the correct answer ("Valentina Tershkova") will not be higher than for a random name. Thus, models do not generalize a prevalent pattern in their training set: if "A is B" occurs, "B is A" is more likely to occur. It is worth noting, however, that if "A is B" appears in-context, models can deduce the reverse relationship. We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. Code available at: https://github.com/lukasberglund/reversal_curse.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
