使用Tuned Lens从Transformer中提取潜在预测
Eliciting Latent Predictions from Transformers with the Tuned Lens
AI总结：
该研究从迭代推理角度分析Transformer，提出改进logit lens的Tuned Lens方法，通过为冻结预训练模型各块训练仿射探针解码隐藏状态，在多模型上验证其更优，并揭示模型预测逐层细化过程。
摘要：
我们从迭代推理的角度分析Transformer，旨在理解模型预测如何逐层细化。为此，我们为冻结的预训练模型中的每个块训练一个仿射探针，使其能够将每个隐藏状态解码为词汇表上的分布。我们的方法Tuned Lens是早期“logit lens”技术的改进，该技术曾带来有用见解但通常很脆弱。我们在多达200亿参数的各种自回归语言模型上测试该方法，表明其比logit lens更具预测性、可靠性且无偏倚。通过因果实验，我们发现Tuned Lens使用与模型本身相似的特征。
英文摘要：
We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer. To do so, we train an affine probe for each block in a frozen pretrained model, making it possible to decode every hidden state into a distribution over the vocabulary. Our method, the tuned lens, is a refinement of the earlier "logit lens" technique, which yielded useful insights but is often brittle. We test our method on various autoregressive language models with up to 20B parameters, showing it to be more predictive, reliable and unbiased than the logit lens. With causal experiments, we show the tuned lens uses similar features to the model itself. We also find the trajectory of latent predictions can be used to detect malicious inputs with high accuracy. All code needed to reproduce our results can be found at https://github.com/AlignmentResearch/tuned-lens.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
