arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.04525cs.CLcs.AI
收藏

语言模型通过共享几何表示和转换概念

Language Models Represent and Transform Concepts with Shared Geometry

Zhimin Hu, Lanhao Niu, Sashank Varma

AI总结:

研究神经网络中概念表示问题,将概念表示形式化为点云流形、上下文转换为向量场,在大语言模型中实例化该框架,发现模型共享概念表示及上下文转换的通用几何结构。

摘要:

概念在神经网络中如何表示是机器学习的基本问题。主流观点将概念表示视为固定几何对象。然而概念出现在上下文中并被其转换。我们从神经群体几何中汲取灵感,将概念表示形式化为点云流形,上下文转换形式化为向量场,并在大语言模型中实例化此框架。

英文摘要:

How concepts are represented in neural networks is a fundamental question in machine learning. The dominant view treats concept representations as stationary geometric objects. Yet concepts appear in context, and context transforms them. Drawing from neural population geometry, we formalize concept representations as point-cloud manifolds and contextual transformations as vector fields, and instantiate this framework in large language models. Across six model families of varying scales, we find that context moves each concept differently. The variance in these displacements is semantically organized, correlating with lexical concreteness and density. Importantly, both the concepts being transformed and this variance structure are shared across models: displacement structure transported from one model predicts held-out displacements in others significantly above chance. Together, these findings show that models share a common geometry not only in how concepts are represented, but more importantly in how context transforms them, a structure with richer organization than prior work has recognized.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑