arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1612.04418cs.IRcs.CLcs.HCcs.LGstat.ML
收藏

基于用户模型的多语言检索意图感知评价

User Model-Based Intent-Aware Metrics for Multilingual Search Evaluation

Yandex

Alexey Drutsa, Andrey Shutovich, Philipp Pushnyakov, Evgeniy Krokhalyov, Gleb Gusev, Pavel Serdyukov

摘要:

尽管网络搜索的多语言方面的重要性越来越大,但到目前为止,还没有合适的离线指标来评估其质量。同时,个人语言偏好可以被视为查询的意图。这种方法将多语言搜索问题转化为搜索多样化的特定任务。此外,可以采用标准的意图感知方法,在经典的IR度量(如ERR)的基础上,为多语言搜索构建多样化的度量。意图感知方法在用户行为模型下估计用户满意度。然而,我们发现,底层的用户行为模型是不现实的,在多语言的情况下,产生的意图感知度量不适当地估计用户满意度。我们开发了一种新的方法来建立意图感知的用户行为模型,克服了这些限制,并转换为质量指标,更好地与标准的在线用户满意度指标。

英文摘要:

Despite the growing importance of multilingual aspect of web search, no appropriate offline metrics to evaluate its quality are proposed so far. At the same time, personal language preferences can be regarded as intents of a query. This approach translates the multilingual search problem into a particular task of search diversification. Furthermore, the standard intent-aware approach could be adopted to build a diversified metric for multilingual search on the basis of a classical IR metric such as ERR. The intent-aware approach estimates user satisfaction under a user behavior model. We show however that the underlying user behavior models is not realistic in the multilingual case, and the produced intent-aware metric do not appropriately estimate the user satisfaction. We develop a novel approach to build intent-aware user behavior models, which overcome these limitations and convert to quality metrics that better correlate with standard online metrics of user satisfaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑