arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.03280cs.AI
收藏

Pythia多跳设置中跨模型激活迁移的负面结果

A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting

Independent Researcher(独立研究者)

Peiyan Zhang, Jason Xin

AI总结:

研究在Pythia-160M到Pythia-410M的多跳推理设置中,通过线性翻译层传递隐藏状态是否能够改善下游回答,结果发现离线表示对齐不足以实现有用的因果通信。

摘要:

最近的研究表明,语言模型可以通过训练过程中生成数据中的隐藏信号传递行为特征。我们提出一个更直接和严格的渠道是否也可行:一个语言模型能否在推理时通过翻译和注入隐藏激活,而不是通过传递自然语言文本,将有用的中间推理状态传递给另一个模型?我们在一个受控的Pythia-160M到Pythia-410M多跳推理设置中测试了这个问题。线性翻译层学习了发送者和接收者隐藏状态之间的强归一化空间映射,归一化余弦相似度在不同种子下接近0.97。然而,当翻译后的激活在推理时注入接收者时,并没有改善下游回答。低强度加法注入仍接近无注入基线,置信区间跨越零。替换式注入始终具有破坏性,将翻译向量重新缩放到接收者隐藏状态范数也无法挽救性能。因此,这是一个有范围的负面结果:在这种设置中,离线表示对齐不足以在接收者内部实现有用的因果通信。

英文摘要:

Recent work shows that language models can transmit behavioural traits through hidden signals in generated data during training. We ask whether a different activation-mediated channel is viable: can one language model communicate a useful intermediate reasoning state to another at inference time through a post-hoc linear activation bridge, rather than through a textual or structured-token relay? We test this question in a controlled Pythia-160M to Pythia-410M multi-hop reasoning setting. A linear translation layer learns a strong normalized-space map between sender and receiver hidden states, with normalized cosine similarity near 0.97 across seeds. However, when the translated activations are injected into the receiver at inference time, they do not improve downstream answering. Low-strength additive injection remains near the no-injection baseline, with confidence intervals that cross zero. Replacement-style injection is consistently destructive, and rescaling translated vectors to the receiver hidden-state norm does not rescue performance. The result is therefore a scoped negative result: in this setting, offline representational alignment is not sufficient for useful causal communication inside the receiver.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑