GraphGhost: 探索大型语言模型背后的结构
GraphGhost: Tracing Structures Behind Large Language Models
AI总结：
GraphGhost通过图分析揭示大型语言模型内部结构，捕捉推理过程中的关键节点与信息流，为理解模型决策提供新视角。
摘要：
大型语言模型（LLMs）在结构化任务上展现出强大的推理能力，但其内部机制仍不为人所知。现有解释方法主要集中在词级归因上，难以深入理解模型内部的多步推理过程。我们提出了GraphGhost，一种基于图的框架，将LLM中的内部词交互和神经元激活建模为图。通过聚合跨层追踪的词依赖关系，GraphGhost捕捉到模型预测背后的全局信息流。我们从两个互补的视角正式化GraphGhost：样本视角，用于追踪个体预测的词依赖关系；数据集视角，用于聚合训练过程中学习到的重复结构模式。通过图分析和定量实验，我们显示图结构属性与重要词和神经元节点密切相关，对结构关键节点的扰动会导致推理行为的可测量变化。这些结果表明，GraphGhost捕获的结构模式反映了LLM推理的有意义内部组织。代码可在软件部分获取。人工制品仅用于研究用途。
英文摘要：
Large Language Models (LLMs) exhibit strong reasoning capabilities on structured tasks, yet the internal mechanisms underlying such behaviors remain poorly understood. Existing interpretation methods mainly focus on token-level attributions, which provide limited insight into multi-step reasoning inside the model. We propose GraphGhost, a graph-based framework that models internal token interactions and neuron activations in LLMs as graphs. By aggregating token dependencies traced across layers, GraphGhost captures global information flow underlying model predictions. We formalize GraphGhost from two complementary perspectives: a sample view, which traces token dependencies for individual predictions, and a dataset view, which aggregates recurring structural patterns learned during training. Through graph analytics and quantitative experiments, we show that graph structural properties are closely associated with influential tokens and neuron nodes, and that perturbations to structurally critical nodes lead to measurable changes in reasoning behavior. These results indicate that the structural patterns captured by GraphGhost reflect meaningful internal organization of LLM reasoning. The codes are available at software part. Artifacts will be made available for research use only.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
