语义欺骗：当推理模型无法计算加法时
Semantic Deception: When Reasoning Models Can't Compute an Addition
AI总结：
研究发现LLMs在面对语义欺骗时表现不佳，揭示其在符号操作上的局限性及对表面语义的依赖。
摘要：
大型语言模型（LLMs）越来越多地用于涉及人类价值观的情境，如涉及推理的人类决策任务。我们通过引入一个实验框架来测试LLMs在处理和操作陌生符号方面的能力，探讨所谓的推理能力。我们引入了语义欺骗：符号由于其形式而具有误导性的语义关联，例如嵌入特定上下文，目的是探测LLMs是否能够维持符号抽象，或者是否默认依赖于已学习的语义关联。我们用新的符号重新定义标准数字和数学运算符，并让LLMs解决用这种改变的符号表达的简单计算。目标是：（1）评估LLMs对任意符号系统的抽象和操作能力；（2）评估其在与任务符号逻辑冲突的误导性语义提示下的能力。通过与四个LLMs的实验，我们发现语义提示可以显著降低推理模型在非常简单的任务上的表现。它们揭示了当前LLMs在符号操作方面的局限性，并突显了一种过度依赖表面语义的趋势，表明思维链可能放大对统计相关性的依赖。即使在LLMs似乎正确遵循指示的情况下，语义提示仍然影响基本能力。这些限制引发了伦理和社会关切，削弱了广泛且有害的将LLMs的推理能力归因于模型的趋势，并说明了LLMs可能在需要稳健符号推理的决策情境中失败，尤其是在不应被模型训练中继承的残余语义关联所妥协的情况下。
英文摘要：
Large language models (LLMs) are increasingly used in situations where human values are at stake, such as decision-making tasks that involve reasoning when performed by humans. We investigate the so-called reasoning capabilities of LLMs over novel symbolic representations by introducing an experimental framework that tests their ability to process and manipulate unfamiliar symbols. We introduce semantic deceptions: situations in which symbols carry misleading semantic associations due to their form, such as being embedded in specific contexts, designed to probe whether LLMs can maintain symbolic abstraction or whether they default to exploiting learned semantic associations. We redefine standard digits and mathematical operators using novel symbols, and task LLMs with solving simple calculations expressed in this altered notation. The objective is: (1) to assess LLMs' capacity for abstraction and manipulation of arbitrary symbol systems; (2) to evaluate their ability to resist misleading semantic cues that conflict with the task's symbolic logic. Through experiments with four LLMs we show that semantic cues can significantly deteriorate reasoning models' performance on very simple tasks. They reveal limitations in current LLMs' ability for symbolic manipulations and highlight a tendency to over-rely on surface-level semantics, suggesting that chain-of-thoughts may amplify reliance on statistical correlations. Even in situations where LLMs seem to correctly follow instructions, semantic cues still impact basic capabilities. These limitations raise ethical and societal concerns, undermining the widespread and pernicious tendency to attribute reasoning abilities to LLMs and suggesting how LLMs might fail, in particular in decision-making contexts where robust symbolic reasoning is essential and should not be compromised by residual semantic associations inherited from the model's training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
