从发现到设计：将概念作为大语言模型的设计轴
From Found to Designed: Concepts as a Design Axis for Large Language Models
浏览论文内容
中文总结 AI 辅助
该研究针对大语言模型概念级结构依赖事后发现而非设计的问题，提出将概念作为设计轴，划分设计空间维度并分析现有模式，推动转向设计带显式概念表示的大语言模型。
中文摘要 AI 辅助
大语言模型（LLMs）编码了丰富的类概念信息，但通过分布式统计关联隐式表示这些信息，而非作为显式、结构化、可组合的概念。因此，概念级结构通常是被发现而非被设计的：它在训练后通过探测或字典学习被恢复，且无架构层面的稳定性、可组合性、可控性或与人类概念组织对齐的保证。我们主张应将概念视为LLMs的设计轴，并沿两个维度划分设计空间：引入概念结构的流水线阶段（训练目标、核心架构、推理或事后解释），以及该结构是源自模型自身表示还是基于外部资源。此分类揭示了三类宽泛模式：推理阶段的方法仍相对未被充分探索，相关想法在流水线各阶段大多孤立发展，外部基于资源的方法虽常以不同术语描述却覆盖整个流水线。这些观察共同推动研究从训练后模型中恢复类概念结构，转向设计带有显式概念表示的LLMs。
英文摘要
Large language models (LLMs) encode rich concept-like information, but represent it implicitly through distributed statistical associations rather than as explicit, structured, compositional concepts. Consequently, concept-level structure is typically \emph{found} rather than \emph{designed}: it is recovered after training through probing or dictionary learning, with no architectural guarantee of stability, compositionality, controllability, or alignment with human conceptual organization. We organize concept-aware interventions along two dimensions: whether concept structure is internally induced or externally grounded, and the stage of the pipeline where it is introduced. This taxonomy reveals three broad patterns: inference-time approaches remain comparatively underexplored, related ideas have developed largely in isolation across pipeline stages, and externally grounded methods span the entire pipeline despite often being described under different terminology. Together, these observations motivate moving beyond recovering concept-like structure from trained models toward designing LLMs with explicit conceptual representations.
发表机构
- Tel Aviv University（特拉维夫大学）
机构由 AI 辅助整理，请以论文原文为准。
