arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.10573cs.CLcs.LG
收藏

大语言模型内部表示中提示的固有维度

The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models

Area Science Park(区域科学园);University of Amsterdam(阿姆斯特丹大学);University of Trieste(的里雅斯特大学)

Karthik Viswanathan, Yuri Gardinazzi, Giada Panerai, Alberto Cazzaniga, Matteo Biagetti

AI总结:

本研究通过固有维度视角分析大语言模型提示的内部表示几何,发现其与下一个 token 不确定性等相关,训练的线性探测模型可在生成前区分恶意与良性提示,准确率优于现有安全工具,为 LLM 安全提供了新信号。

摘要:

我们通过固有维度的视角研究大语言模型中提示级别的 token 表示几何。将 Transformer 视为平均场粒子系统,我们估计每一层经验测度的固有维度,并证明其与下一个 token 的不确定性相关。在不同模型和固有维度估计器中,我们发现固有维度在早期到中间层达到峰值,并且在句法和语义干扰(通过打乱 token)下会增加,同时它与平均惊奇值(surprisal)密切相关,通过 softmax 可将 logits 几何与熵建立简单关联。作为可解释性和安全性的实际案例研究,我们基于每一层的固有维度轮廓训练线性探测模型,以在生成前区分恶意提示和良性提示。该探测模型在不同数据集上的准确率达到 90% 至 95%,优于 Llama Guard 和 Shield Gemma 等广泛使用的安全护栏。我们进一步将其与通过 Tuned Lens 得到的分层熵构建的线性探测模型进行比较,发现基于固有维度的探测模型具有竞争力且互补,提供了一种分布在各层的紧凑、可解释的信号。我们的研究结果表明,提示级几何为监测和控制 LLM 行为提供了可操作的信号,并为机制见解与实际安全工具之间搭建了桥梁。

英文摘要:

We study the geometry of token representations at the prompt level in large language models through the lens of intrinsic dimension. Viewing transformers as mean-field particle systems, we estimate the intrinsic dimension of the empirical measure at each layer and demonstrate that it correlates with next-token uncertainty. Across models and intrinsic dimension estimators, we find that intrinsic dimension peaks in early to middle layers and increases under syntactic and semantic disruption (by shuffling tokens), and that it is strongly correlated with average surprisal, with a simple analysis linking logits geometry to entropy via softmax. As a case study in practical interpretability and safety, we train a linear probe on the per-layer intrinsic dimension profile to distinguish malicious from benign prompts before generation. This probe achieves accuracy of 90 to 95\% in different datasets, outperforming widely used guardrails such as Llama Guard and Shield Gemma. We further compare against linear probes built from layerwise entropy derived via the Tuned Lens and find that the intrinsic dimension-based probe is competitive and complementary, offering a compact, interpretable signal distributed across layers. Our findings suggest that prompt-level geometry provides actionable signals for monitoring and controlling LLM behavior, and offers a bridge between mechanistic insights and practical safety tools.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑