通过反向训练来应对逆转诅咒
Reverse Training to Nurse the Reversal Curse
AI总结：
针对大语言模型的逆转诅咒，本文提出反转训练字符串但保留实体等子串的反向训练方案，在标准任务和逆转任务上均取得更优表现。
摘要：
大语言模型（LLM）存在一个令人惊讶的失败：当在“A具有特征B”上进行训练时，它们无法泛化到“B是A的特征”，这被称为逆转诅咒。即使使用数万亿个token进行训练，由于Zipf定律，这个问题仍然会出现——因此，即便我们在整个互联网上进行训练也是如此。本工作提出了一种替代训练方案，称为反向训练，其中所有词都被使用两次，使可用token数量翻倍。LLM通过反转训练字符串而在正向和反向上都得到训练，同时保留（即不反转）选定的子串，例如实体。我们表明，数据匹配的反向训练模型在标准任务上提供了优于标准模型的性能，而计算匹配的反向训练模型在逆转任务上提供了远为优越的性能，有助于解决逆转诅咒问题。
英文摘要：
Large language models (LLMs) have a surprising failure: when trained on "A has a feature B", they do not generalize to "B is a feature of A", which is termed the Reversal Curse. Even when training with trillions of tokens this issue still appears due to Zipf's law - hence even if we train on the entire internet. This work proposes an alternative training scheme, called reverse training, whereby all words are used twice, doubling the amount of available tokens. The LLM is trained in both forward and reverse directions by reversing the training strings while preserving (i.e., not reversing) chosen substrings, such as entities. We show that data-matched reverse-trained models provide superior performance to standard models on standard tasks, and compute-matched reverse-trained models provide far superior performance on reversal tasks, helping resolve the reversal curse issue.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
