评估交叉编码器在心理问卷语义相似性评估中的应用
Evaluating cross-encoders for semantic similarity assessment in psychological questionnaires
- Institute of Psychology, University of Innsbruck（因斯布鲁克大学心理学研究所）
- University of Applied Sciences Kufstein, Tyrol（库夫施泰因应用科学大学）
- Psychiatry and Psychotherapy Clinic, University of Ulm（乌尔姆大学精神病学与心理治疗诊所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究以NEO-FFI和PID5BF+M数据为基础，探讨交叉编码器检测问卷条目语义重叠的效果，发现其语义距离与条目相关性负相关，但预测价值更多依赖模型训练特征而非编码器架构。
AI中文摘要：
评分量表间的相关性通常被解释为收敛效度或区分效度的证据，但此前研究表明，这些关联部分可能归因于问卷条目措辞间的语义相似性，而非仅由真实的构念重叠导致。基于主要源自双编码器的相关证据，本研究探讨联合编码条目对的交叉编码器是否为检测问卷条目间语义重叠的合适技术。利用NEO-FFI和PID5BF+M的作答数据（样本量N=502，Labek等人，2024），我们检验了交叉编码器得出的语义相似性估计值是否与经验条目相关性相关，以及交叉编码器是否比双编码器具有系统性优势。在12种交叉编码器模型中，语义距离与绝对条目相关性始终呈负相关，其中三分之二的模型达到统计显著性，R²值最高达0.37。然而，基于相同基础模型的交叉编码器并未始终优于双编码器。这些发现将量表间相关性中存在语义成分的先前证据扩展至交叉编码器架构，同时表明预测价值更多取决于模型特定的训练特征，而非编码器架构本身。
英文摘要：
Correlations between rating scales are commonly interpreted as evidence of convergent or discriminant validity, yet prior studies suggest that part of these associations may be attributable to semantic similarity between item wordings rather than to genuine construct overlap alone. Building on this evidence, largely derived from bi-encoders, the present study explores whether cross-encoders, which jointly encode item pairs, offer a suitable technique for detecting semantic overlapping between questionnaire items. Using response data from the NEO-FFI and the PID5BF+M (N = 502, Labek et al., 2024), we examined whether cross-encoder-derived semantic similarity estimates are associated with empirical item correlations, and whether cross-encoders offer a systematic advantage over bi-encoders. Across twelve cross-encoder models, semantic distance was consistently negatively associated with absolute item correlations, reaching statistical significance in two-thirds of the models, with R2 values of up to .37. However, cross-encoders did not consistently outperform bi-encoders based on the same base models. These findings extend prior evidence for semantic components in scale intercorrelations to cross-encoder architectures, while indicating that predictive value depends more on model-specific training characteristics than on encoder architecture itself.
