arXiv 2604.041822026-04-05cs.AI

比较性反向学习揭示在非平稳不确定性下LLMs的刚性适应

Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty

Insight Research Ireland Centre for Data Analytics, Ireland（爱尔兰洞察研究中心）；School of Computing, Dublin City University, Ireland（都柏林城市大学计算机学院）

Haomiaomiao Wang, Tomás E Ward, Lili Zhang

备注：: 14 pages, 2 figures, accepted by IPMU 2026, SS04: Explainable AI and Decision-Making Under Uncertainty: Bridging Interpretability and Robustness

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究通过比较确定性固定转移循环与随机性随机调度，探讨LLMs在非平稳环境下的适应性，发现DeepSeek-V3.2表现出极端的固着现象，而Gemini-3和GPT-5.2适应更快但仍不如人类敏感。

摘要：

非平稳环境要求智能体在 contingencies 变化时修正先前学习的动作值。我们将大语言模型（LLMs）视为在两个选项概率反向学习任务中的序列决策策略，包含三个潜在状态和由性能标准或超时触发的切换事件。我们比较了确定性固定转移循环与增加波动性的随机调度，并评估了DeepSeek-V3.2、Gemini-3和GPT-5.2，以人类数据为行为参考。在所有模型中，win-stay接近天花板，而lose-shift明显减弱，揭示了正负证据的非对称使用。DeepSeek-V3.2在反向后表现出极端的固着和弱学习，而Gemini-3和GPT-5.2适应更快但仍比人类更不敏感于损失。随机转移增强了LLMs的反向特定持久性，但并未均匀减少总胜利，表明高总收益可以与刚性适应共存。层次强化学习（RL）拟合表明存在可区分的机制：刚性可能源于弱损失学习、政策确定性膨胀或通过反事实抑制的价值极化。这些结果推动了反向敏感诊断和考虑波动性的LLM评估模型。

英文摘要：

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。