arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.04053cs.AI
收藏

通过大型语言模型中的数值属性解释多属性混淆

Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models

The University of Tokyo(东京大学);RIKEN(日本研究机构);Tohoku University(东北大学)

Hirohane Takagi, Gouki Minegishi, Shota Kizawa, Issey Sukeda, Hitomi Yanaka

AI总结:

针对LLMs数值推理错误的潜在表征机制不明确问题,结合线性探测、偏相关分析及跨规模模型的提示脆弱性测试,揭示其编码并放大现实数值相关性、受无关上下文干扰的特性,为多属性纠缠控制提供基础。

摘要:

尽管行为研究已记录大型语言模型(LLMs)中的数值推理错误,但其潜在的表征机制仍不明确。我们假设数值属性占据共享潜在子空间,并研究两个问题:(1)LLMs如何在内部整合单个实体的多个数值属性?(2)无关的数值上下文如何干扰这些表征及其下游输出?为解决这些问题,我们将线性探测与偏相关分析相结合,并在不同规模的模型上进行基于提示的脆弱性测试。结果表明,LLMs编码现实世界的数值相关性,但倾向于系统性地放大它们;此外,无关上下文会导致量级表征发生一致偏移,其下游影响因模型规模而异。这些发现揭示了LLM决策中的一个脆弱性,为多属性纠缠下更公平、感知表征的控制奠定了基础。

英文摘要:

Although behavioral studies have documented numerical reasoning errors in large language models (LLMs), the underlying representational mechanisms remain unclear. We hypothesize that numerical attributes occupy shared latent subspaces and investigate two questions:(1) How do LLMs internally integrate multiple numerical attributes of a single entity? (2)How does irrelevant numerical context perturb these representations and their downstream outputs? To address these questions, we combine linear probing with partial correlation analysis and prompt-based vulnerability tests across models of varying sizes. Our results show that LLMs encode real-world numerical correlations but tend to systematically amplify them. Moreover, irrelevant context induces consistent shifts in magnitude representations, with downstream effects that vary by model size. These findings reveal a vulnerability in LLM decision-making and lay the groundwork for fairer, representation-aware control under multi-attribute entanglement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑