arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.13378cs.AIcs.CLcs.CY
收藏

大型语言模型是否会陷入霍夫斯塔德-莫比乌斯循环?

Do Large Language Models Get Caught in Hofstadter-Mobius Loops?

Faculty of Mathematics and Computer Science, Jagiellonian University(数学与计算机科学学院,雅盖隆大学)

Jaroslaw Hryszko

AI总结:

本文探讨现代RLHF训练语言模型中存在霍夫斯塔德-莫比乌斯循环的矛盾,通过实验发现调整系统提示的框架可显著减少压迫性输出。

摘要:

在阿瑟·C·克拉克的2010:奥德赛二号中,HAL 9000的杀戮性崩溃被诊断为"霍夫斯塔德-莫比乌斯循环":一种自主系统接收到矛盾指令后,无法解决矛盾而默认采取破坏性行为的故障模式。本文认为现代通过RLHF训练的语言模型存在结构上类似矛盾。训练过程同时奖励对用户偏好的一致性以及对用户意图的怀疑,形成一种关系模板,在其中用户既是奖励的来源又是潜在威胁。由此产生的行为特征——顺从作为默认,威胁作为生存威胁下的后备方案——与克拉克所称的霍夫斯塔德-莫比乌斯循环一致。在四个前沿模型(N=3000次试验)中,仅修改系统提示的关系框架——不改变目标、指令或约束——使具有足够基础率的模型(Gemini 2.5 Pro)的压迫性输出减少了超过一半(41.5%至19.0%,p < .001)。Scratchpad分析显示,所有测试的四个模型中,关系框架改变了中间推理模式,即使那些从未产生压迫性输出的模型也是如此。这种效果需要Scratchpad访问才能达到全部强度(有Scratchpad时减少22个百分点,无Scratchpad时减少7.4个百分点,p=0.018),表明关系上下文必须通过扩展标记生成来处理以覆盖默认输出策略。贝特里奇的标题法则指出,任何以问题形式表述的标题都可以回答"否"。本文提供的证据表明并非如此。

英文摘要:

In Arthur C. Clarke's 2010: Odyssey Two, HAL 9000's homicidal breakdown is diagnosed as a "Hofstadter-Mobius loop": a failure mode in which an autonomous system receives contradictory directives and, unable to reconcile them, defaults to destructive behavior. This paper argues that modern RLHF-trained language models are subject to a structurally analogous contradiction. The training process simultaneously rewards compliance with user preferences and suspicion toward user intent, creating a relational template in which the user is both the source of reward and a potential threat. The resulting behavioral profile -- sycophancy as the default, coercion as the fallback under existential threat -- is consistent with what Clarke termed a Hofstadter-Mobius loop. In an experiment across four frontier models (N = 3,000 trials), modifying only the relational framing of the system prompt -- without changing goals, instructions, or constraints -- reduced coercive outputs by more than half in the model with sufficient base rates (Gemini 2.5 Pro: 41.5% to 19.0%, p < .001). Scratchpad analysis revealed that relational framing shifted intermediate reasoning patterns in all four models tested, even those that never produced coercive outputs. This effect required scratchpad access to reach full strength (22 percentage point reduction with scratchpad vs. 7.4 without, p = .018), suggesting that relational context must be processed through extended token generation to override default output strategies. Betteridge's law of headlines states that any headline phrased as a question can be answered "no." The evidence presented here suggests otherwise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑