短视域与稀疏概念：J-lens读出的数学视角

中文摘要 AI 辅助

Jacobian lens（J-lens）已被提出作为从语言模型中读出可言语化表征的一种方式，但其原理和含义缺乏详细的理论探讨。我们对这种解释及其假定的因果结构提供了数学视角。除了将J-lens视为一种启发式探针外，我们还进一步将其视为从中间激活到预期未来读出的一阶因果转移算子。我们将Jacobian矩阵作为下游映射的最优局部线性近似进行研究，分析其全局近似行为与偏差，并确定其数学含义为对预期未来读出的期望。对Jacobian能量分布的进一步分析表明，其因果几何结构高度稀疏：能量随深度衰减，集中在极小比例中，并分解为对角路径和特定关键位置。这种分解进一步将J-lens对未来输出的期望解析为短视域与稀疏概念预测，为J-lens在思考过程中可视化概念的能力提供了更直观的归因与解释。基于该理论，我们提出了一种简单但有效的J-lens改进策略与解耦方法，显著增强了J-lens读出正确中间概念的能力。

英文摘要

The Jacobian lens (J-lens) has been proposed as a way to read verbalizable representations from language models. However, its principle and meaning lack a detailed and theoretical discussion. We provide a mathematical view of this interpretation and of its assumed causal structure. Besides treating the J-lens as a heuristic probe, we further regard it as a first-order causal transfer operator from intermediate activations to expected future readouts. We study the Jacobian matrix as the optimal local linear approximation of the downstream mapping, analyze its global approximation behavior and bias, and identify its mathematical meaning as an expectation over anticipated future readouts. Further analysis of the Jacobian energy distribution reveals that its causal geometry is highly sparse. The energy decays with depth, concentrates in an extremely small proportion, and decomposes into diagonal pathways and specific critical positions. This decomposition further resolves the expectation of the J-lens over future outputs into short-horizon and sparse concept predictions, providing a more intuitive attribution and explanation for the ability of the J-lens to visualize concepts during the thinking process. Based on the theory, we propose a simple but effective improvement strategy and decoupling method for the J-lens, which significantly enhances the ability of the J-lens to read out correct intermediate concepts.

发表机构

• Alibaba Group（阿里巴巴集团）
• USTC（中国科学技术大学）

机构由 AI 辅助整理，请以论文原文为准。