arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.06963cs.CLcs.AIcs.LG
收藏

下一词元预测的陷阱

The pitfalls of next-token prediction

ETH Zürich, Switzerland(苏黎世联邦理工学院);Google Research, US(谷歌研究院)

Gregor Bachmann, Vaishnavh Nagarajan

AI总结:

该研究指出下一词元预测中教师强制训练本身存在无法学习准确预测器的深层问题,通过最小规划任务验证了Transformer和Mamba架构的相关失效,并提出无教师训练的多词元预测方法作为解决方案。

摘要:

仅靠下一词元预测器能否忠实地模拟人类智能?我们明确了这一新兴担忧,纠正了围绕它的普遍误解,并提出了一种简单的多词元目标。\n首先,我们认为下一词元预测中两个常被混淆的阶段——自回归推理与教师强制训练——必须被明确区分。关于自回归推理过程中误差会累积的流行批评,关键前提是假设教师强制训练已经学习到了准确的下一词元预测器。这一假设回避了我们揭示的一个更根深蒂固的问题:在某些类别的任务中,教师强制训练从一开始就可能无法学习到准确的下一词元预测器。我们描述了教师强制训练失效的一般机制,并设计了一个最小规划任务,实验中Transformer和Mamba架构都以这种方式失效——值得注意的是,该任务本身是很容易学习的。\n最后,我们提供了初步证据,表明这种失效可以通过无教师训练解决,这是一种使用虚拟标记的简单改进方法,可提前预测多个词元。我们希望这一发现能为未来的讨论奠定基础,并激发对下一词元预测范式之外方向的探索。我们的代码已公开于https://github.com/gregorbachmann/Next-Token-Failures。

英文摘要:

Can a mere next-token predictor faithfully model human intelligence? We crystallize this emerging concern and correct popular misconceptions surrounding it, and advocate a simple multi-token objective. As a starting point, we argue that the two often-conflated phases of next-token prediction -- autoregressive inference and teacher-forced training -- must be treated distinctly. The popular criticism that errors can compound during autoregressive inference, crucially assumes that teacher-forcing has learned an accurate next-token predictor. This assumption sidesteps a more deep-rooted problem we expose: in certain classes of tasks, teacher-forcing can simply fail to learn an accurate next-token predictor in the first place. We describe a general mechanism of how teacher-forcing can fail, and design a minimal planning task where both the Transformer and the Mamba architecture empirically fail in that manner -- remarkably, despite the task being straightforward to learn. Finally, we provide preliminary evidence that this failure can be resolved using _teacherless_ training, a simple modification using dummy tokens that predicts multiple tokens in advance. We hope this finding can ground future debates and inspire explorations beyond the next-token prediction paradigm. We make our code available under https://github.com/gregorbachmann/Next-Token-Failures

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑