他们在想什么？LLM中概念的界定、探测与追踪
What are They Thinking? Delineation, Probing, and Tracking of Concepts in LLMs
AI总结：
本文提出通过线性探针低成本地检测LLM嵌入中的概念，并展示了概念界定、探针训练与跨上下文追踪的方法，为大规模模型监控奠定基础。
摘要：
随着LLM影响力的扩大，深入了解其决策变得至关重要。一种方法是开发探针，用于检测LLM计算的嵌入中是否存在广泛的概念——这可以说是模型在“思考”的内容。这些探针应成本低廉且易于应用于任何LLM，以便在正常操作期间能够监控多个概念。在本文中，我们通过定义并执行关键任务的示例，迈出了开发创建此类探针能力的第一步：首先，通过创建包含概念存在和不存在的数据集来仔细界定概念；然后，训练并测试一组线性探针，以检测LLM任何层上的概念，包括对所需探针复杂性的探索；最后，我们展示了此类探针可以跨更大上下文追踪概念。我们使用四个不同的概念和三个不同的LLM进行了实验。当这一过程扩展到更多概念时，将能够轻松监控新模型。
英文摘要：
As the influence of LLMs expands, it is imperative to gain insight into their decisions. One way to do that is to develop probes that detect the presence or absence of a broad set of high-level abstract concepts within the embeddings computed in an LLM - which is what we might say a model is ``thinking" about. Such probes should be low-cost and easily applicable to any LLM, so that monitoring for many concepts is possible during normal operation. In this paper, we take the first steps towards developing the capability of creating many such probes by defining and executing examples of the key tasks needed: first, the careful delineation of a high-level abstract concept through the creation of a dataset with the concept both present and then absent. Then, the training and testing of a set of linear probes to detect the concept on any layer of an LLM, including an exploration of the complexity of the probe needed. Finally, we show that such probes can track concepts across larger contexts. This is done with four separate concepts and three different LLMs. When this process is scaled to many more concepts, it will create the ability to monitor new models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
