均值池化余弦相似度并非长度不变：理论和跨领域证据支持一种长度不变的替代方案
Mean-Pooled Cosine Similarity is Not Length-Invariant: Theory and Cross-Domain Evidence for a Length-Invariant Alternative
AI总结：
研究指出均值池化余弦相似度在现代Transformer表示中并非长度不变，通过实验验证了其在跨语言Python接近性中的解释力，并提出中心核对齐（CKA）作为更优的替代方法。
摘要：
均值池化余弦相似度是跨语言、模态和任务比较神经表示的默认度量。我们证明该度量并非长度不变：在现代Transformer表示中，均值池化余弦相似度随序列长度单调增加，与表示内容无关。实证上，在HumanEvalPack上四个代码LLM中，长度比单独解释了跨语言“Python接近性”的R²=0.52-0.75，而AST深度和共享令牌分数仅贡献不到3%的解释方差。用中心核对齐（CKA）会减少解释方差83%，并反转长度系数的符号（β_len: +0.86 → -0.37）。在Mistral-7B上平行WMT对中，余弦相似度的R²分别为EN-FR 0.23，EN-DE 0.33，而CKA的R²低于0.01。在CLIP ViT-B/32中，均值池化相比EOS池化减少了长度效应（R²: 0.21 → <0.01），这与理论对各向异性依赖一致。我们主张长度不变的度量如CKA应成为跨表示比较的默认选择，且最近基于均值池化余弦相似度的跨语言表示收敛声称值得重新审视。
英文摘要：
Mean-pooled cosine similarity is the default metric for comparing neural representations across languages, modalities, and tasks. We establish that this metric is not length-invariant: under the anisotropy that characterizes modern transformer representations, mean-pooled cosine grows monotonically in sequence length, independent of representational content. Empirically, on HumanEvalPack across four code LLMs, the length ratio alone explains $R^2 = 0.52$--$0.75$ of cross-language "Python proximity," while AST depth and shared-token fraction add less than 3% of explained variance beyond length. Substituting Centered Kernel Alignment (CKA) reduces explained variance by 83% and reverses the sign of the length coefficient ($β_{\mathrm{len}}: +0.86 \to -0.37$). The same pattern holds in Mistral-7B on parallel WMT pairs ($R^2 = 0.23$ EN-FR, $R^2 = 0.33$ EN-DE for cosine; $R^2 < 0.01$ for CKA). In CLIP ViT-B/32, mean-pooling reduces the length effect relative to EOS-pooling ($R^2: 0.21 \to {<}0.01$), as predicted by the theory's dependence on anisotropy. We argue that length-invariant metrics such as CKA should be the default for cross-representation comparisons, and that recent claims of cross-lingual representational convergence built on mean-pooled cosine warrant re-examination.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
