arXiv 2601.228182026-07-20cs.CRcs.AI

嵌入空间中的隐秘行动：基于几何的隐写术与大语言模型中的检测

Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models

UCL Centre for Artificial Intelligence, University College London, UK（伦敦大学学院人工智能中心，大学学院伦敦）；School of Computing and Communications, Lancaster University, UK（兰卡斯特大学计算机与通讯学院）；Department of Informatics, University of Sussex, UK（苏塞克斯大学信息学院）；Centre for Psychedelic Research and Centre for Complexity Science, Imperial College London, UK（伦敦帝国学院迷幻研究与复杂科学中心）；Centre for Eudaimonia and Human Flourishing, University of Oxford, UK（牛津大学幸福与人类繁荣中心）

Charles Westphal, Keivan Navaie, Fernando E. Rosas

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出了一种低恢复性隐写术，通过嵌入空间衍生映射提升秘密恢复率，同时减少负载恢复性，并提出基于机制可解释性的检测方法，提高微调模型中的秘密检测准确率。

摘要：

经过微调的LLMs可以通过隐写通道在输出中隐蔽地编码提示秘密。先前的工作展示了这种威胁，但依赖于可以轻易恢复的编码。我们通过分类器准确性正式化负载恢复性，并显示先前的方案实现了100%的恢复性。对此，我们引入了低恢复性隐写术，用嵌入空间衍生的映射取代任意映射。对于在TrojanStego提示上训练的Llama-8B（LoRA）和Ministral-8B（LoRA），精确秘密恢复率从17%上升到30%（+78%），从24%上升到43%（+80%），而在在Wiki提示上训练的Llama-70B（LoRA）中，从9%上升到19%（+123%），同时减少负载恢复性。我们随后讨论检测。我们认为，检测基于微调的隐写攻击需要超越传统隐写分析的方法。标准方法测量分布偏移，这是微调的预期副作用。相反，我们提出了一种机制可解释性方法：在后期层激活上训练的线性探针可以检测秘密，其在微调模型中的检测准确率比基模型高33%以上，即使对于低恢复性方案也是如此。这表明恶意微调留下了可解释性防御的可操作内部签名。

英文摘要：

Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17$\rightarrow$30\% (+78\%) and 24$\rightarrow$43\% (+80\%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9$\rightarrow$19\% (+123\%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33\% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。