ColBERT的基于[MASK]的查询增强:将查询输入长度增加至四倍的影响
ColBERT's [MASK]-based Query Augmentation: Effects of Quadrupling the Query Input Length
AI总结:
本文研究了ColBERT中基于[MASK]标记的查询增强机制,发现将查询长度扩展至四倍对检索性能影响甚微,证明了模型对超长查询输入的鲁棒性。
摘要:
ColBERT的一个独特之处在于它使用[MASK]标记对查询进行评分(查询增强)。先前的研究表明,[MASK]标记对非[MASK]查询词进行加权,强调某些词而不是最初提出的引入全新的词。我们首先证明,先前报道的ColBERTv1中[MASK]标记的词加权行为在ColBERTv2中同样成立。然后,我们研究了将[MASK]标记数量从零增加到训练时使用的查询输入长度的四倍的效果,包括第一阶段检索和候选评分,观察到在少量[MASK]时性能最初下降,当添加足够的[MASK]将查询填充到32的平均长度时性能大幅提升,随后性能趋于平稳。此外,我们将基线性能与查询长度扩展到128个标记时的性能进行比较,发现差异很小(例如,在各种指标上相差不到1%),且通常在统计上不显著,这表明如果ColBERT遇到比预期更多的[MASK]标记,性能不会崩溃。
英文摘要:
A unique aspect of ColBERT is its use of [MASK] tokens in queries to score documents (query augmentation). Prior work shows [MASK] tokens weighting non-[MASK] query terms, emphasizing certain tokens over others , rather than introducing whole new terms as initially proposed. We begin by demonstrating that a term weighting behavior previously reported for [MASK] tokens in ColBERTv1 holds for ColBERTv2. We then examine the effect of changing the number of [MASK] tokens from zero to up to four times past the query input length used in training, both for first stage retrieval, and for scoring candidates, observing an initial decrease in performance with few [MASK]s, a large increase when enough [MASK]s are added to pad queries to an average length of 32, then a plateau in performance afterwards. Additionally, we compare baseline performance to performance when the query length is extended to 128 tokens, and find that differences are small (e.g., within 1% on various metrics) and generally statistically insignificant, indicating performance does not collapse if ColBERT is presented with more [MASK] tokens than expected.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。