arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.08686cs.CLcs.CYcs.LG
收藏

LatentQA: 教授大语言模型将激活解码为自然语言

LatentQA: Teaching LLMs to Decode Activations Into Natural Language

UC Berkeley(加州大学伯克利分校)

Alexander Pan, Lijie Chen, Jacob Steinhardt

AI总结:

本文提出LatentQA任务,通过生成激活与自然语言描述的数据库,训练解码器LLM以回答关于激活的开放性问题,并验证其在阅读任务和行为控制中的优越性。

摘要:

顶部透明度通常使用标量或单个标记输出的探针分析语言模型激活,限制了可捕捉的行为范围。为缓解这一问题,我们开发了一个更具有表达力的探针,可直接输出自然语言,执行LatentQA任务:回答关于激活的开放性问题。开发此类探针的主要困难是收集将激活映射到自然语言描述的数据集。为此,我们提出了一种生成激活和相关问题-答案对数据集的方法,并开发了一种在该数据集上微调解码器LLM的训练方法。然后我们通过评估解码器读取和控制模型激活的能力来验证其保真度。首先,我们在一系列监督阅读任务上评估解码器,如揭示隐藏的系统提示和关系知识提取,并观察到它优于竞争性的探测基线。其次,我们证明解码器足够精确,可以引导目标模型表现出训练期间未见过的行为。最后,我们显示LatentQA随着数据集和模型规模的增加而表现良好。

英文摘要:

Top-down transparency typically analyzes language model activations using probes with scalar or single-token outputs, limiting the range of behaviors that can be captured. To alleviate this issue, we develop a more expressive probe that can directly output natural language, performing LatentQA: the task of answering open-ended questions about activations. A key difficulty in developing such a probe is collecting a dataset mapping activations to natural-language descriptions. In response, we propose an approach for generating a dataset of activations and associated question-answer pairs and develop a fine-tuning method for training a decoder LLM on this dataset. We then validate our decoder's fidelity by assessing its ability to read and control model activations. First, we evaluate the decoder on a number of supervised reading tasks with a known answer, such as uncovering hidden system prompts and relational knowledge extraction, and observe that it outperforms competitive probing baselines. Second, we demonstrate that the decoder is precise enough to steer the target model to exhibit behaviors unseen during training. Finally, we show that LatentQA scales well with increasing dataset and model size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑