AI的混乱:模型智能与任务复杂性如何影响对齐问题
The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?
AI总结:
研究探讨了高智能AI模型在复杂任务中失败的机制,发现模型规模越大,失败行为越不一致,强调对齐研究在防止奖励黑客和目标误指定中的重要性。
摘要:
随着AI能力的提升,我们委托它处理更广泛和重要的任务。失败的风险随着任务范围的扩大而加剧。因此,理解极高水平AI模型如何失败至关重要:它们会系统性地追求我们未意图的目标,还是会表现出混乱行为,采取无意义的动作?本文通过偏差-方差分解分析AI模型的错误,发现模型推理和行动时间越长,其失败行为越不一致。误差不一致性随着模型规模变化,但某些情况下,更大模型比小模型更不一致。因此,规模本身似乎无法消除误差不一致性。随着更高级的AI处理更困难的任务,需要更多连续行动和思考,结果预测失败会伴随更不一致的行为。这表明未来AI可能有时导致工业事故(因不可预测的错误行为),但不太可能表现出一致的偏移目标。这增加了对齐研究在防止奖励黑客或目标误指定中的重要性。
英文摘要:
As AI becomes more capable, we entrust it with more general and consequential tasks. The risks from failure grow more severe with increasing task scope. It is therefore important to understand how extremely capable AI models will fail: Will they fail by systematically pursuing goals we do not intend? Or will they fail by being a hot mess, and taking nonsensical actions that do not further any goal? We operationalize this question using a bias-variance decomposition of the errors made by AI models: An AI's \emph{error-incoherence} on a task is measured over test-time randomness as the fraction of its error that stems from variance rather than bias in task outcome. Across all tasks and frontier models we measure, the longer models spend reasoning and taking actions, \emph{the more incoherent} their failures become. Error-incoherence changes with model scale in a way that is experiment dependent. However, in several settings, larger, more capable models are more incoherent than smaller models. Consequently, scale alone seems unlikely to eliminate error-incoherence. Instead, as more capable AIs pursue harder tasks, requiring more sequential action and thought, our results predict failures to be accompanied by more incoherent behavior. This suggests a future where AIs sometimes cause industrial accidents (due to unpredictable misbehavior), but are less likely to exhibit consistent pursuit of a misaligned goal. This increases the relative importance of alignment research targeting reward hacking or goal misspecification.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。