当LLM学会一致错误：合成欺骗的线性表示的多模型研究
Linear Separability of Activation Representations after Supervised Fine-Tuning on Incorrect Responses: A Study of Synthetic Dishonesty in Large Language Models
AI总结：
通过LoRA微调五个Transformer模型的诚实与欺骗变体，使用线性探针检测合成欺骗，发现早期层即可达到近完美AUC，支持线性表示假说，并揭示两种表示机制。
摘要：
欺骗性对齐（模型保持准确的内部表示同时故意产生错误输出）仍然是AI安全的核心挑战。虽然战略性欺骗是主要的长期关注点，但通过直接优化错误答案诱导的合成不诚实为研究学习欺骗的表示基础提供了受控测试平台。我们引入了一个多模型范式，其中五个Transformer模型（Pythia-1.4B、Gemma-2-2B/9B、Qwen2.5-7B、Llama-3.1-8B）的诚实和欺骗变体使用LoRA在相同问题分布上进行微调。在平均池化隐藏状态上训练的线性探针在四个架构的1-3层即可检测到合成欺骗，AUC接近完美（≥0.99），而Pythia-1.4B达到峰值0.705。逻辑回归探针始终匹配或优于MLP探针，支持线性表示假说。在TruthfulQA上训练的探针以近乎零损失（ΔAUC≈0）泛化到保留的MMLU主题。深层表示对高斯噪声表现出强鲁棒性，其中Gemma-2模型表现出卓越的稳定性。对Fisher判别比、有效秩、质心几何、方向稳定性、跨域对齐和校准（ECE）的机制分析揭示了两种机制：Pythia/Llama/Qwen中的表示坍缩与Gemma-2中的高维保持。在所有模型中，欺骗方向在更深层逐渐巩固，在1-4层可实现最优校准（除Pythia外ECE<0.01）。这些结果表明，通过适度的监督微调，鲁棒、域不变的欺骗表示可以迅速固化，对基于激活的监控具有启示意义。
英文摘要：
When a language model is fine-tuned to produce systematically incorrect responses, does this training leave a structured, linearly recoverable trace in its internal activations? We study this question in a controlled model-organism setting using five transformer architectures spanning 1.4 to 9 billion parameters. For each model, an "honest" and a "dishonest" LoRA fine-tuned variant are constructed using identical question distributions but correct versus plausible-but-incorrect answers. Linear probes reach near-ceiling separability (AUC >= 0.9997) within the first few layers in four of five architectures. This separability transfers from TruthfulQA to held-out MMLU subjects for four models, but substantially less so for Pythia-1.4B. Six geometric analyses reveal an architectural dichotomy in representation structure. A control experiment further shows that LoRA fine-tuning itself leaves a strong activation fingerprint that is largely dissociable from the honesty-related signal, independent of fine-tuning data content. Two supplementary experiments qualify these findings: the identified direction transfers poorly to paraphrased questions, with AUC approaching chance, and norm-calibrated activation steering provides no reliable evidence of a causal role in generation, partly because stronger interventions disrupt output fluency. These results characterize what supervised fine-tuning on incorrect responses induces in activation space while clarifying important limitations on interpretation and causal significance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
