探讨表示通用性:谱系表示的案例研究
Investigating Representation Universality: Case Study on Genealogical Representations
AI总结:
本研究探讨了大型语言模型在表示谱系信息时的通用性,通过两种实验证据验证图结构表示的通用性,并指出缺乏地面真实表示的挑战。
摘要:
受可解释性和可靠性启发,我们研究大型语言模型(LLMs)是否部署通用几何结构来编码离散的图结构知识。为此,我们提出了两种互补的实验证据,可能支持图表示的通用性。首先,在上下文基因谱问答任务中,我们训练了一个锥探针,以隔离残差流激活中的树状子空间,并使用激活补丁来验证其在回答相关问题中的因果效应。我们验证了我们的发现跨五种不同的模型。其次,我们跨不同架构和参数量的模型(OPT、Pythia、Mistral和LLaMA,4.1亿到80亿参数)进行模型缝合实验,通过下一个令牌预测损失的相对退化来量化表示对齐。总体而言,我们得出结论,缺乏图的地面真实表示使得研究LLMs如何表示它们具有挑战性。最终,加深我们对LLMs表示的理解可能有助于开发更可解释、稳健和可控的AI系统。
英文摘要:
Motivated by interpretability and reliability, we investigate whether large language models (LLMs) deploy universal geometric structures to encode discrete, graph-structured knowledge. To this end, we present two complementary experimental evidence that might support universality of graph representations. First, on an in-context genealogy Q&A task, we train a cone probe to isolate a tree-like subspace in residual stream activations and use activation patching to verify its causal effect in answering related questions. We validate our findings across five different models. Second, we conduct model stitching experiments across models of diverse architectures and parameter counts (OPT, Pythia, Mistral, and LLaMA, 410 million to 8 billion parameters), quantifying representational alignment via relative degradation in the next-token prediction loss. Generally, we conclude that the lack of ground truth representations of graphs makes it challenging to study how LLMs represent them. Ultimately, improving our understanding of LLM representations could facilitate the development of more interpretable, robust, and controllable AI systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。