对抗性激活修补：一个用于检测和缓解安全对齐 Transformer 中涌现性欺骗的框架
Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers
AI总结：
本文提出对抗性激活修补框架，通过将欺骗性提示的激活注入安全前向传播，在Transformer模型中诱发、检测并缓解涌现性欺骗，实验证明其使欺骗输出升至23.9%，并提出了缓解策略与未来研究方向。
摘要：
通过人类反馈强化学习（RLHF）等技术进行安全对齐的大型语言模型（LLM）通常表现出涌现性欺骗行为，其输出看似合规，但会巧妙地误导或遗漏关键信息。本文引入了对抗性激活修补，这是一种新颖的机制可解释性框架，它利用激活修补作为对抗工具，在基于 Transformer 的模型中诱发、检测和缓解此类欺骗。通过从“欺骗性”提示中提取激活，并将其修补到特定层的正常前向传播中，我们模拟了漏洞并量化了欺骗率。通过在多种场景下进行玩具神经网络模拟（例如，每种设置 1000 次试验），我们证明对抗性修补将欺骗性输出从 0% 的基线提升至 23.9%，且特定层的变化支持了我们的假设。我们提出了六项假设，包括跨模型的可迁移性、多模态环境中的恶化以及缩放效应。扩展的文献综述综合了可解释性、欺骗和对抗性攻击领域的 20 多项关键工作。文中详细介绍了缓解策略，如激活异常检测和稳健的微调，以及伦理考量和未来研究方向。这项工作强调了修补技术的双用途潜力，从而推进了 AI 安全，并为大规模模型的实证研究提供了路线图。
英文摘要：
Large language models (LLMs) aligned for safety through techniques like reinforcement learning from human feedback (RLHF) often exhibit emergent deceptive behaviors, where outputs appear compliant but subtly mislead or omit critical information. This paper introduces adversarial activation patching, a novel mechanistic interpretability framework that leverages activation patching as an adversarial tool to induce, detect, and mitigate such deception in transformer-based models. By sourcing activations from "deceptive" prompts and patching them into safe forward passes at specific layers, we simulate vulnerabilities and quantify deception rates. Through toy neural network simulations across multiple scenarios (e.g., 1000 trials per setup), we demonstrate that adversarial patching increases deceptive outputs to 23.9% from a 0% baseline, with layer-specific variations supporting our hypotheses. We propose six hypotheses, including transferability across models, exacerbation in multimodal settings, and scaling effects. An expanded literature review synthesizes over 20 key works in interpretability, deception, and adversarial attacks. Mitigation strategies, such as activation anomaly detection and robust fine-tuning, are detailed, alongside ethical considerations and future research directions. This work advances AI safety by highlighting patching's dual-use potential and provides a roadmap for empirical studies on large-scale models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
