不同分词器丰富度下跨语言的概念方向可靠性

中文摘要 AI 辅助

提取的情感方向在不同样本间可能存在差异，即使下游情感分类仍然准确。为了评估方向的可复现性，我们在四种语言模型上，使用母语文本和翻译文本，测量了英语、豪萨语和约鲁巴语表示中的分裂半一致性。我们使用十个主题识别用于一致性评估的层，并评估了十五个主题的独立分组之间的方向一致性。使用最终令牌，英语的分裂半一致性范围为0.737至0.870，豪萨语为0.589至0.762，约鲁巴语为0.101至0.399，在所有77个完整模型比较中保持此语言排名顺序。在这些相同层上训练的分类器始终以高于偶然的概率预测情感，表明预测准确性并不暗示方向一致性。此外，对令牌表示进行平均会产生较不一致的一致性，且高一致性可能部分反映句子长度。最终，我们的发现强调了独立于分类性能来衡量向量方向可复现性的必要性，尽管并未确立分词器丰富度（即每个空白分隔词的平均令牌数）导致跨语言差异。

英文摘要

Extracted sentiment directions can vary across samples even when downstream sentiment classification remains accurate. To evaluate direction reproducibility, we measure split-half agreement in English, Hausa, and Yoruba representations across four language models using both native and translated texts. We identify layers selected for agreement using ten topics and evaluate direction agreement across separate groups of fifteen topics. Using the final token, split-half agreement ranges from 0.737 to 0.870 for English, 0.589 to 0.762 for Hausa, and 0.101 to 0.399 for Yoruba, maintaining this language rank order across all 77 complete model comparisons. Classifiers trained on these same layers consistently predict sentiment above chance, demonstrating that predictive accuracy does not imply directional consistency. Furthermore, averaging token representations yields less consistent agreement, and high agreement can partially reflect sentence length. Ultimately, our findings highlight the need to measure vector direction reproducibility independently of classification performance, though they do not establish that tokenizer fertility which is the average number of tokens per whitespace separated word causes cross-lingual differences.

发表机构

• University of Cape Town（开普敦大学）
• African Institute for Mathematical Sciences（非洲数学科学研究所）
• Bayero University Kano（卡诺巴耶罗大学）
• Federal University Dutse（联邦杜策大学）
• University of Vienna（维也纳大学）

机构由 AI 辅助整理，请以论文原文为准。