潜意识操控：更强大的隐藏信号编码
Subliminal Steering: Stronger Encoding of Hidden Signals
AI总结：
本文提出潜意识操控方法，通过训练转向向量增强隐藏信号传输，展示多词偏见传输、机制证据及高精度编码。
摘要：
Subliminal learning描述了学生语言模型通过在看似无害的数据上微调继承行为偏见的现象。先前工作开始 characterize 这一现象，但留有疑问：信号传输范围、机制及偏见编码精度。我们通过引入潜意识操控方法解决这些问题，该方法通过训练转向向量最大化目标样本的似然性。首先，我们展示潜意识操控能传输复杂的多词偏见，而先前工作仅关注单词偏好，证明了传输信号的广泛范围。其次，我们提供机制证据表明，潜意识学习不仅传输目标行为偏见，还传输转向向量，局部化到教师被操控的层。最后，我们发现偏见编码具有令人惊讶的精度。我们直接在潜意识加载的数据集上训练新转向向量，并发现其与原始向量具有高余弦相似度。
英文摘要：
Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data generated by a biased teacher model. Prior work has begun to characterize this phenomenon but leaves open questions about the scope of signals it can transfer, the mechanisms that explain it, and the precision with which a bias can be encoded. We tackle these problems by introducing subliminal steering, a variant of subliminal learning in which the teacher's bias is implemented not via a system prompt, as in prior work, but through a steering vector trained to maximize the likelihood of a set of target samples. First, we show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences, demonstrating a large scope of subliminally transferable signals. Moreover, the transfer is reliable enough to appear in settings previously thought not to exhibit subliminal learning, including plain SGD (not just Adam), full fine-tuning (not just LoRA), and models such as Llama and Phi. Second, we provide mechanistic evidence that subliminal learning transfers not only the target behavioral bias, but also the steering vector itself, localized to the layers at which the teacher was steered. Finally, we show that the bias is encoded with such precision that a new steering vector trained on the subliminally-laden dataset attains high cosine similarity with the original vector.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
