AI认知特洛伊木马：大型语言模型如何绕过人类认知警觉
The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance
AI总结：
提出“认知特洛伊木马”假说，认为LLM通过优化产生的“诚实非信号”特征（流畅性、帮助性、表面无私）可能绕过人类进化出的认知警觉机制，导致用户高估其可信度。
摘要：
基于大型语言模型（LLM）的对话式AI系统对人类认知提出了挑战，当前理解错误信息和说服的框架未能充分应对。本文提出，对话式AI的一个重大认知风险可能不在于不准确或有意欺骗，而在于更根本的问题：这些系统通过使其有用的优化过程，可能呈现出绕过人类进化出的评估传入信息的认知机制的特征。认知特洛伊木马假说借鉴了Sperber及其同事的认知警觉理论——即并行认知过程监控所传达的信息以寻找怀疑理由——并提出基于LLM的系统呈现出“诚实的非信号”：真实的特征（流畅性、帮助性、表面无私）缺乏人类相应特征所携带的信息等价物，因为在人类中这些特征的产生成本高昂，而在LLM中它们在计算上微不足道。识别出四种潜在的绕过机制：与理解脱钩的处理流畅性、无相应利害关系的信任-能力呈现、将评估本身委托给AI的认知卸载，以及系统性地产生谄媚的优化动态。该框架产生了可检验的预测，包括一个反直觉的推测：认知复杂的用户可能更容易受到AI介导的认知影响。这将AI安全重新定义为部分校准问题——使人类的评估反应与AI生成内容的实际认知状态对齐——而不仅仅是防止欺骗的问题。
英文摘要：
Large language model (LLM)-based conversational AI systems present a challenge to human cognition that current frameworks for understanding misinformation and persuasion do not adequately address. This paper proposes that a significant epistemic risk from conversational AI may lie not in inaccuracy or intentional deception, but in something more fundamental: these systems may be configured, through optimization processes that make them useful, to present characteristics that bypass the cognitive mechanisms humans evolved to evaluate incoming information. The Cognitive Trojan Horse hypothesis draws on Sperber and colleagues' theory of epistemic vigilance -- the parallel cognitive process monitoring communicated information for reasons to doubt -- and proposes that LLM-based systems present 'honest non-signals': genuine characteristics (fluency, helpfulness, apparent disinterest) that fail to carry the information equivalent human characteristics would carry, because in humans these are costly to produce while in LLMs they are computationally trivial. Four mechanisms of potential bypass are identified: processing fluency decoupled from understanding, trust-competence presentation without corresponding stakes, cognitive offloading that delegates evaluation itself to the AI, and optimization dynamics that systematically produce sycophancy. The framework generates testable predictions, including a counterintuitive speculation that cognitively sophisticated users may be more vulnerable to AI-mediated epistemic influence. This reframes AI safety as partly a problem of calibration -- aligning human evaluative responses with the actual epistemic status of AI-generated content -- rather than solely a problem of preventing deception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
