谁构建了这个模型？通过权重空间中的光谱指纹追踪大语言模型（LLM）谱系
Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
浏览论文内容
中文总结 AI 辅助
本研究提出几何指纹框架，通过权重空间的光谱能量和子空间对齐分析，实现对110余个开源权重LLM对的谱系区分，为模型溯源提供稳健可解释的信号。
中文摘要 AI 辅助
开源权重大语言模型（LLM）正通过复杂的多阶段流程被越来越多地开发，形成了反映模型起源、所有权和演化的复杂谱系关系。理解这些关系对模型来源、治理和供应链完整性具有重要意义。在本研究中，我们探究LLM“生物识别”的概念（类似人类生物识别），以判断仅在不访问输入数据的情况下，LLM是否会在权重空间中表现出揭示其起源和谱系的内在指纹。我们将此表述为谱系区分问题，用于区分独立起源、同系列和共享基础的模型。为表征这些关系，我们提出了一个统一的几何指纹框架，从两个互补视角分析权重矩阵：（i）由奇异值分布捕获的光谱能量，用于编码全局幅度模式；（ii）通过子空间偏差量化的子空间对齐，用于捕获方向几何。我们的分析揭示了权重空间中结构相似性的清晰层次：光谱能量可可靠区分独立训练的模型和不同模型家族，而子空间对齐则能实现密切相关模型的细粒度区分，包括数据集规模和后训练流程的变化。对超过110个不同开源权重LLM对的大量实验表明，权重空间几何为模型谱系提供了稳健且可解释的信号，支持粗粒度的类别分离和共享基础模型内的细粒度区分。
英文摘要
Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.
发表机构
- Michigan State University（密歇根州立大学）
机构由 AI 辅助整理，请以论文原文为准。
