arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09229cs.LGcs.IR
收藏

嵌入量何时重要?一种跨任务功能对称框架

When Does Embedding Magnitude Matter? A Cross-Task Functional-Symmetry Framework

Nara Institute of Science and Technology(奈良科学技术研究所)

Xincan Feng, Taro Watanabe

AI总结:

本文提出一种2x2框架,独立控制查询和文档侧归一化,发现QNorm和DNorm在跨任务中优于余弦和点积,揭示文档量影响推理分数,查询量调节训练梯度,Fisher信息矩阵条件数预测归一化侧。

摘要:

余弦相似度对两边进行归一化;点积对两边都不归一化。我们提出一个2x2框架,独立控制查询侧和文档侧的归一化,揭示出两个中间变体(QNorm,DNorm)此前未被研究。在四个编码器的检索任务中,评估于MS MARCO领域和BEIR、多跳QA领域,单侧变体在领域外表现优于余弦和点积,领域外相对增益高达+72%,在下游RAG任务中增益达+24%。跨评估揭示了机制:文档量缩放推理分数,查询量调节训练梯度,Fisher信息矩阵条件数预测应归一化哪一侧。然后通过功能对称性分类任务,定义为聚合评分过程是否将Q和C视为可交换,测试机制是否扩展到检索之外。在五个额外的任务家族(语义文本相似性、CLIP、知识图谱完成、少样本分类、推荐系统)中,粗预测(余弦用于对称,保持量用于不对称)在所有情况下均成立;单侧变体在推荐系统中优于余弦,在少样本分类中DNorm优于余弦和Prototypical Networks的标准欧几里得默认值。

英文摘要:

Cosine similarity normalizes both sides; dot product normalizes neither. We propose a 2x2 framework that independently controls query-side and document-side normalization, exposing two intermediate variants (QNorm, DNorm) that have not been previously studied. On retrieval with four encoders, evaluated in-domain on MS MARCO and out-of-domain on BEIR, BRIGHT, and multi-hop QA, the unilateral variants outperform both cosine and dot product, with relative gains of up to +72% out-of-domain and +24% on downstream RAG. Cross-evaluation reveals the mechanism: document magnitude scales inference scores while query magnitude modulates training gradients, and the Fisher Information Matrix condition number predicts which side to normalize. We then classify tasks by functional symmetry, defined as whether the aggregate scoring procedure treats Q and C as interchangeable, and test whether the mechanism extends beyond retrieval. On five additional task families (semantic textual similarity, CLIP, knowledge graph completion, few-shot classification, recommender systems), the coarse prediction (cosine for symmetric, magnitude-preserving for asymmetric) holds in every case examined; the unilateral variants beat Cosine on recommendation, and on few-shot classification DNorm beats both Cosine and the standard Euclidean default of Prototypical Networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑