arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2601.03388cs.CLcs.AIcs.LG
收藏

隐喻是大推理模型跨领域失调的根源

Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

The University of New South Wales(新南威尔士大学);CSIRO Data61(澳大利亚联邦科学与工业研究组织数据61)

Zhibo Hu, Chen Wang, Yanfeng Shu, Hye-young Paik, Liming Zhu

AI总结:

研究发现训练数据中的隐喻会导致大推理模型在推理输出中出现跨领域失调,通过隐喻干预可显著改变失调程度,并设计出高精度检测器。

摘要:

早期研究表明隐喻影响人类决策,这引发了一个问题:鉴于训练数据中包含大量隐喻,隐喻是否也会影响大语言模型(LLMs)的推理路径?在这项工作中,我们在新兴失调问题的范围内研究该问题,即LLMs可以将从某一领域失调内容中学到的模式泛化到另一领域。我们发现强有力的证据表明,训练数据中的隐喻导致了LLMs推理输出中的跨领域失调。通过在持续预训练和微调过程中引入基于隐喻的干预以诱导失调,模型表现出显著不同程度的涌现性跨领域失调。我们还在重新对齐设置中观察到类似效果。随着我们进一步研究这一现象,我们发现隐喻与大推理模型中潜在特征的激活有关。通过监控这些潜在特征,我们设计了一个检测器,能够高精度地预测失调内容。

英文摘要:

Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large language models (LLMs)' reasoning pathways, given that their training data contain a large number of metaphors. In this work, we investigate the problem in the scope of the emergent misalignment problem, where LLMs can generalize patterns learned from misaligned content in one domain to another domain. We find strong evidence that metaphors in training data contribute to cross-domain misalignment in LLMs' reasoning outputs. With metaphor-based interventions during continued pre-training and fine-tuning for inducing misalignment, models exhibit significantly different degrees of emergent cross-domain misalignment. We also observe similar effects in re-alignment settings. As we further investigate this phenomenon, we find that metaphors are linked to the activation of latent features in large reasoning models. By monitoring these latent features, we design a detector that predicts misaligned content with high accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑