arXiv 2605.309812026-05-29cs.CLcs.LG

自回归Transformer中的认知疲劳：形式化与测量

Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement

Guru Gobind Singh Indraprastha University, India（古鲁·戈宾德·辛格·印度普拉斯塔大学）；Artificial Intelligence Institute, University of South Carolina, USA（人工智能研究所，南卡罗来纳大学）；Indian Institute of Technology, Kanpur, India（印度理工学院，坎浦尔）；Indian AI Research Organization, India（印度人工智能研究组织）

Riju Marwah, Ritvik Garimella, Vishal Pallagani, Atishay Jain, Michael Stewart, Amit Sheth

备注：: 9 pages, 7 figures. Accepted at the 43rd International Conference on Machine Learning (ICML 2026)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文形式化自回归语言模型在长程生成中的退化现象为认知疲劳，并提出轻量级诊断指标疲劳指数（FI），通过聚合注意力衰减、表征漂移和熵校准三个信号实现实时监测，实验表明FI能高精度预测任务退化和重复生成。

摘要：

自回归语言模型在长程生成过程中经常退化，产生重复文本、失去指令遵循能力并表现出不稳定的熵。尽管这些失败普遍存在，但从业者缺乏在线诊断工具来实时检测它们。我们将这种退化形式化为认知疲劳，这是一种可测量的生成时状态，其特征是对原始提示的注意力衰减、表征漂移和熵校准错误。我们引入了疲劳指数（FI），这是一种轻量级、模型无关的诊断方法，在明确的公理（单调性、有界性、可解释性）下聚合这三个信号，从而实现可靠的运行时监控。在九个模型（1B-13B参数）上，FI轨迹表现出结构化的时间动态，预测任务退化（AUROC = 0.95）和重复（Spearman rho = 0.94），并揭示了非单调的缩放行为：低于3B的指令微调模型比基础模型退化更快，而在7B时这一趋势逆转。压力分析进一步表明，在更长的上下文、中间位置的证据和降低的数值精度下，FI onset加速。这些结果确立了认知疲劳作为一个连贯且可测量的现象，并将FI定位为生产级LLM系统中运行时可靠性监控的原则性工具。

英文摘要：

Autoregressive language models frequently degrade during long-horizon generation, producing repetitive text, losing instruction adherence, and exhibiting unstable entropy. Despite the prevalence of these failures, practitioners lack online diagnostics to detect them in real-time as they occur. We formalize this degradation as cognitive fatigue, a measurable generation-time state characterized by decay in attention to the original prompt, representational drift, and entropy miscalibration. We introduce the Fatigue Index (FI), a lightweight, model-agnostic diagnostic that aggregates these three signals under explicit axioms (monotonicity, boundedness, interpretability) enabling reliable runtime monitoring. Across nine models (1B-13B parameters), FI trajectories exhibit structured temporal dynamics, predict task degradation (AUROC = 0.95) and repetition (Spearman rho = 0.94), and reveal non-monotonic scaling behavior: instruction-tuned models below 3B exhibit faster collapse than base models, with this trend reversing at 7B. Stress analyses further show that FI onset accelerates under longer contexts, middle-positioned evidence, and reduced numerical precision. These results establish cognitive fatigue as a coherent and measurable phenomenon, and position FI as a principled tool for runtime reliability monitoring in production LLM systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。