arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.09714cs.CLcs.AIcs.LG
收藏

全是代码,没有思考:当前语言模型在密码化语言中难以推理

All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Language

Anthropic Fellows Program(Anthropic Fellow项目);Constellation;Anthropic

Shiyuan Guo, Henry Sleight, Fabien Roger

AI总结:

本研究测试28种密码下模型的推理能力,发现密码化推理能力与预训练数据中密码普遍性相关,且逃避思维链监控对当前模型可能无效。

摘要:

随着AI代理的普及,检测有害的AI行为变得重要。思维链(CoT)监控是广泛用于检测对抗性攻击和AI错位的方法之一。然而,攻击者和错位模型可能通过密码化推理(即隐藏在加密、翻译或压缩文本中的推理)来逃避CoT监控。为评估此风险,我们测试模型是否能执行密码化推理。对于28种不同的密码,我们对多达10个模型进行微调和提示,使其以该密码进行推理。我们以数学问题的模型准确率作为推理能力的代理指标。在我们测试的模型中,我们发现一种不对称性:即使在模型通过准确翻译成英文展示了对密码文本的理解能力,当以密码文本推理时,模型准确率可能显著下降。即使是前沿模型在较少见的密码上也表现挣扎,尽管它们能在rot13等知名密码上准确推理。我们表明,密码化推理能力与预训练数据中密码的普遍性相关。我们还发现了缩放定律,显示密码化推理能力随着额外微调数据的增加而缓慢提升。我们的工作表明,使用密码化推理逃避CoT监控对当前模型可能是一种无效策略,并为限制未来前沿模型中该能力的发展提供了指导。

英文摘要:

Detecting harmful AI actions is important as AI agents gain adoption. Chain-of-thought (CoT) monitoring is one method widely used to detect adversarial attacks and AI misalignment. However, attackers and misaligned models might evade CoT monitoring through ciphered reasoning: reasoning hidden in encrypted, translated, or compressed text. To assess this risk, we test whether models can perform ciphered reasoning. For each of 28 different ciphers, we fine-tune and prompt up to 10 models to reason in that cipher. We measure model accuracy on math problems as a proxy for reasoning ability. Across the models we test, we find an asymmetry: model accuracy can drop significantly when reasoning in ciphered text, even though models demonstrate comprehension of ciphered text by being able to translate it accurately to English. Even frontier models struggle with lesser-known ciphers, although they can reason accurately in well-known ciphers like rot13. We show that ciphered reasoning capability correlates with cipher prevalence in pretraining data. We also identify scaling laws showing that ciphered reasoning capability improves slowly with additional fine-tuning data. Our work suggests that evading CoT monitoring using ciphered reasoning may be an ineffective tactic for current models and offers guidance on constraining the development of this capability in future frontier models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑