超越静态几何的潜意识提示：因果深度与多词元混淆
Subliminal Prompting Beyond Static Geometry: Causal Depth and Multi-Token Confounds
浏览论文内容
中文总结 AI 辅助
本研究通过固定动物-数字提示协议，分别测量输出共变、向量对齐、隐藏状态可读性和因果控制，发现固定几何、因果时序与多词元测量是独立属性，并揭示长度混淆，约束了词元级解释。
中文摘要 AI 辅助
潜意识学习表明，语言模型可以通过看似无关的输出传递隐藏特征。一种提出的解释——词元纠缠——通过模型的输出词汇表将动物和数字词元联系起来。然而，现有的测量回答了不同的问题：输出是否共变、固定输出向量是否对齐、能否从隐藏状态读出答案、或者该状态是否因果地控制答案。我们在一个固定的动物-数字提示协议中分别测量了每一项。从Llama-3.1-8B到70B，固定输出向量相似性对行为的预测能力较差：配对平均相关变化为-0.080（95%置信区间[-0.127, -0.035]）。固定输出头读出在归一化深度AUC上未显示显著变化。为了测试控制，我们将一个数字提示中临时答案位置的状态复制到另一个提示的五个深度，并测量最终动物分数跟随哪个提示。供体控制AUC从0.254升至0.540，配对变化为+0.286（95%置信区间[+0.272, +0.300]），所有18个概念均有增加。在恰好剩余八个Transformer块时，对比仍然存在，而特异性和身份对照保持较小或精确。在两个Qwen模型中，按顺序对每个数字进行评分并未恢复正向的单词元关联。逐词元平均反而产生了正向的合并关联，但在控制数字宽度后消失，揭示了长度混淆。因此，固定几何、可观察性读出、因果时序和多词元测量是这一冻结提示通道的不同属性。它们约束了词元级解释，但并未确定训练时特征传递的机制。
英文摘要
Subliminal learning shows that language models can transmit a hidden trait through outputs that appear unrelated to it. One proposed explanation, token entanglement, links animal and number tokens through the model's output vocabulary. Yet existing measurements answer different questions: whether outputs co-vary, fixed output vectors align, an answer can be read from a hidden state, or that state causally controls the answer. We measure each separately in a fixed animal-number prompting protocol. From Llama-3.1-8B to 70B, fixed output-vector similarity predicts behavior less well: the paired mean correlation change is -0.080 (95% CI [-0.127, -0.035]). A fixed output-head readout shows no resolved change in normalized depth AUC. To test control, we copy the temporary answer-position state from one number prompt into another at five depths and measure which prompt the final animal score follows. Donor-control AUC rises from 0.254 to 0.540, a paired change of +0.286 (95% CI [+0.272, +0.300]), with increases for all 18 concepts. The contrast remains with exactly eight transformer blocks remaining, while specificity and identity controls remain small or exact. In two Qwen models, scoring every digit in sequence does not recover the positive one-token association. Per-token averaging instead creates a positive pooled association that disappears after controlling number width, revealing a length confound. Thus, fixed geometry, observational readability, causal timing, and multi-token measurement are distinct properties of this frozen prompting channel. They constrain token-level explanations but do not identify the mechanism of training-time trait transfer.
