函数保持重参数化下的表示测量
Representation Measurements Under Function-Preserving Reparameterizations
- University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究指出列排列并行分析违反函数保持重参数化不变性，经多模型、多域实验验证其分量计数与决策易受参考分布影响，正交不变比较器则更稳定。
AI中文摘要：
隐藏坐标并非由语言模型的输入-输出函数唯一确定，因此源自表示的测量应在函数保持基变换下保持不变。本研究表明，列排列并行分析违反函数保持重参数化不变性，因为当模型函数和观测协方差谱保持固定时，其参考分布和所选分量数量会发生变化。更普遍地说，数据内部参考过程无法同时保持每个坐标边际、保持正交等变性以及去除跨坐标协方差。实验上，在5个模型、3个检索域和75种变换中，分量数量的中位数不一致为0.79，固定阈值决策的中位数不一致为0.26。仅中心化的对照实验分离出参考驱动效应，尽管观测谱未变，1200个分量数量中有1141个发生变化，而独立并行分析种子不会改变任何对应决策。相比之下，正交不变比较器分数在保持相似保留判别力的同时数值稳定。这些结果共同表明，并行分析得到的分量数量和决策可反映隐藏坐标选择，而非模型的明确定义属性。
英文摘要：
Hidden coordinates are not uniquely determined by a language model's input--output function, so representation-derived measurements should be invariant to function-preserving changes of basis. This study shows that column-permutation parallel analysis violates function-preserving reparameterization invariance because its reference distribution and selected component count can change while the model function and observed covariance spectrum remain fixed. More generally, a data-internal reference procedure cannot simultaneously preserve every coordinate marginal, remain orthogonally equivariant, and remove cross-coordinate covariance. Empirically, across five models, three retrieval domains, and 75 transformations, median component-count disagreement is 0.79 and median fixed-threshold decision disagreement is 0.26. A centering-only control isolates the reference-driven effect, with 1,141 of 1,200 component counts changing despite an unchanged observed spectrum, whereas independent parallel analysis seeds change none of the corresponding decisions. By contrast, orthogonally invariant comparator scores remain numerically stable with similar held-out discrimination. Together, these results show that parallel analysis-derived component counts and decisions can reflect hidden-coordinate choice rather than a well-defined property of the model.
