arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.01381cs.IRcs.LG
收藏

连续使用样本的稳定和半稳定抽样方法

Stable and Semi-stable Sampling Approaches for Continuously Used Samples

Microsoft(微软)

Nikita Astrakhantsev, Deepak Chittajallu, Nabeel Kaushal, Vladislav Mokeev

摘要:

信息检索系统通常通过标记与用户查询样本相对应的结果的相关性来测量。在实际的搜索引擎中,这种测量需要连续地执行,例如每天或每周。这在(a)查询样本对产品当前查询流量的代表性;(b)标签成本:如果我们保持相同的查询样本,结果将是相似的,允许我们重用它们的标签;以及(c)由连续使用相同查询样本引起的过拟合之间创建了权衡。在本文中,我们明确制定这种权衡,提出了两个新的变种-稳定和半稳定-简单和加权随机抽样,并表明他们优于现有的方法的连续使用设置,包括监测/调试搜索引擎或比较排名候选人。

英文摘要:

Information retrieval systems are usually measured by labeling the relevance of results corresponding to a sample of user queries. In practical search engines, such measurement needs to be performed continuously, such as daily or weekly. This creates a trade-off between (a) representativeness of query sample to current query traffic of the product; (b) labeling cost: if we keep the same query sample, results would be similar allowing us to reuse their labels; and (c) overfitting caused by continuous usage of same query sample. In this paper we explicitly formulate this tradeoff, propose two new variants -- Stable and Semi-stable -- to simple and weighted random sampling and show that they outperform existing approaches for the continuous usage settings, including monitoring/debugging search engine or comparing ranker candidates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑