Chain-of-thought (CoT) monitoring is a promising safety mechanism for AI agents, based on the premise that visible reasoning traces can surface misaligned or deceptive behavior. While effective in standard scenarios, recent work highlights that LLMs remain vulnerable to persuasion-based jailbreaks, where natural-language arguments override model constraints. We stress-test whether this vulnerability extends to monitoring LLMs: can an adversarial agent persuade its CoT monitor to approve proposed actions that violate the monitor's policy? We design an evaluation framework with 40 tasks and analyze thousands of agent-monitor interactions, where agents are instructed to argue for policy-violating proposals. We find that in such adversarial settings, monitor access to the agent's CoT reasoning increases rather than decreases approval of harmful actions on average by 9.5%, as the scratchpad provides an additional persuasion channel. To address this, we introduce a fact-checking monitoring framework. We find that a fact-checker and monitor pairing from different model families, for example a Claude 3.7 Sonnet monitor paired with a GPT-4.1 fact-checker, reduces approval of policy-violating actions by up to 45%, compared to only 6%, when using the same model for both fact-checking and monitoring roles. Our results demonstrate that CoT monitoring alone may be insufficient against adversarial persuasion, and that model-diverse fact-checking provides a robust mitigation.
Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure
Digital Epidemiology Laboratory, Digital Futures, University of Cincinnati(数字流行病学实验室,数字未来,辛辛那提大学);Center for Humanities and Technology (CHaT), University of Cincinnati(人文与科技中心(CHaT),辛辛那提大学);Norwegian University of Science and Technology(挪威科技大学)
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
UK AI Security Institute(英国人工智能安全研究所);Apollo Research(阿波罗研究);METR;University of Montreal(蒙特利尔大学);Mila;Anthropic;OpenAI(开放人工智能研究所);Google DeepMind(谷歌DeepMind);Truthful AI;UC Berkeley(伯克利大学);Center for AI Safety(人工智能安全中心);AI Futures Project(人工智能未来项目);Amazon(亚马逊);Scale AI;Magic;Meta;Redwood Research(红木研究)
Bandung Fe Institute & Adjunct Science Fellow in InaAI(巴旦格Fe研究所及InaAI兼职科学研究员);AI Research Center IT Del & Bandung Fe Institute(IT Del人工智能研究中心及巴旦格Fe研究所)
Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);ICISEE, Shanghai Jiao Tong University(上海交通大学ICISEE);School of Mathematical Sciences, Institute of Natural Sciences, MOE-LSC, CMA-Shanghai, Shanghai Jiao Tong University(上海交通大学数学科学学院);King Abdullah University of Science and Technology(卡塔尔国王 Abdullah 科学与技术大学)