基础模型在随机性与创造性上胜过对齐模型
Base Models Beat Aligned Models at Randomness and Creativity
AI总结：
研究指出LLM对齐技术不应普遍应用，发现在随机数生成、混合策略博弈和创意写作等需要不可预测输出的任务上，基础模型持续优于对齐模型，且常见基准表现与这些任务表现存在权衡。
摘要：
对齐已迅速成为LLM开发中的默认要素，诸如基于人类反馈的强化学习等技术使模型能够安全运作、遵循指令，并在复杂任务上表现更佳。尽管这些技术确有其用，我们提出不应将其普遍应用，并展示了一系列基础语言模型持续优于其流行对齐版本的任务。具体而言，我们研究了需要不可预测输出的任务，例如随机数生成、混合策略博弈（石头剪刀布和捉迷藏）以及创意写作。在每种情况下，对齐模型都倾向于狭隘的行为模式，从而导致明显的劣势，例如相较于其他均匀随机数更偏好生成“7”，在某些博弈状态下变得几乎完全可预测，或者将讨喜的写作置于创意原创性之上。在所有测试模型中，在常见基准上的表现越好，往往与我们任务上的表现越差相关，这表明所需能力之间存在一种有效的权衡。
英文摘要：
Alignment has quickly become a default ingredient in LLM development, with techniques such as reinforcement learning from human feedback making models act safely, follow instructions, and perform ever-better on complex tasks. While these techniques are certainly useful, we propose that they should not be universally applied and demonstrate a range of tasks on which base language models consistently outperform their popular aligned forms. Particularly, we study tasks that require unpredictable outputs, such as random number generation, mixed strategy games (rock-paper-scissors and hide-and-seek), and creative writing. In each case, aligned models tend towards narrow behaviors that result in distinct disadvantages, for instance, preferring to generate "7" over other uniformly random numbers, becoming almost fully predictable in some game states, or prioritizing pleasant writing over creative originality. Across models tested, better performance on common benchmarks tends to correlate with worse performance on our tasks, suggesting an effective trade-off in the required capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
