各向同性悬崖：大语言模型决策的几何特征
Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models
浏览论文内容
中文总结 AI 辅助
该研究从各向同性视角分析大语言模型多项选择题问答的决策几何，识别出各向同性转变的决策关键过渡层，发现该几何行为与下游准确率高度相关且对提示变化鲁棒，揭示模型决策的通用机制。
中文摘要 AI 辅助
我们从各向同性的视角研究多项选择题问答（MCQA）中决策的几何特性。分析五个开放权重模型在不同数据集上的表现，我们识别出各向同性发生转变的决策关键过渡层，这与主要表征变化及任务相关聚类的出现相吻合。我们证明这种同步几何行为与下游准确率高度相关（r≈0.84），显示其对成功决策的重要性。此外，我们表明该过渡对提示变化具有鲁棒性，提示这反映了模型行为的通用机制。
英文摘要
We investigate the geometry of decision-making in Multiple Choice Question Answering (MCQA) through the lens of isotropy. Analyzing five open-weight models across diverse datasets, we identify decision-critical transition layers characterized by a shift in isotropy, coinciding with a major representational change and the emergence of task-relevant clusters. We demonstrate that this synchronized geometric behavior is strongly correlated with downstream accuracy ($r\approx0.84$), displaying its relevance for successful decision-making. Furthermore, we show that this transition is robust to prompt variations, suggesting that it reflects a general mechanism of model behavior.
发表机构
- University of Tübingen（蒂宾根大学）
- The University of Texas at Austin（德克萨斯大学奥斯汀分校）
机构由 AI 辅助整理，请以论文原文为准。
