arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.08130cs.IR
收藏

基于行为的会话满意度评估

Behavior-based evaluation of session satisfaction

Chao Liu, Zhenzhen Zheng, Jinkang Jia

摘要:

如今,网络搜索在世界范围内变得越来越流行。许多研究人员和开发人员对搜索用户的行为进行了大量的研究。在实践中,对这些行为的充分认识,不仅有助于评估新开发的排序算法的有用性和搜索引擎的其他变化,而且可以指导搜索引擎的发展方向。据我们所知,过去的研究大多集中在单个搜索引擎的评价上,这在早期确实促进了搜索引擎的快速发展。然而,这些页面级别的行为是如此有限,以至于不能再对搜索引擎的微小变化给出明确的反馈。我们认为,当提供更多有关搜索会话的信息时,它将更加准确和敏感。本文提出了一种会话级评估方法。检索并仔细分析会话级功能。在评价模型中选取了反映最终满意度的线性和非线性特征。设计并训练了一个具有不同粒度的两层混合评价模型,该模型能达到较好的查准率和查全率。通过大量的实际实验进行评估,结果表明,该模型取得了更高的准确率性能比传统的页面级评估指标。此外,为了实际应用,解释每个会话的满意度判断的原因是重要的。本文提出了一个会话级的评价模型,并将其应用于搜索引擎公司的实际应用中。

英文摘要:

Nowadays, web search becomes more and more popular all over the world. Many researchers and developers have done lots of studies on behaviors of search users. In practice, the full understanding of these behaviors can not only help to evaluate the usefulness of newly-developed ranking algorithms and other changes of search engine, but also to guide the growth direction of search engine. As far as we know, most of past work are mainly focused on single search evaluation, which do promote the rapid development of search engine in early stage. However,these page-level behaviors are so limited that can no longer give explicit feedbacks on minor changes of the search engine. We think that it will be more accurate and sensitive when more information on search session are provided. In this paper, a session level evaluation method is proposed. The session-level features are retrieved and carefully analyzed. Some linear and non-linear features which can reflect the final degree of satisfaction are chosen and adopted in evaluation models. A two-layer hybrid evaluation model with different granularity, which can achieve good precision and recall, is designed and trained. Lots of real experiments are evaluated by the model, the result shows it achieved a higher accuracy performance than traditional page-level evaluation metrics. Furthermore, for practical application, it is important to interpret the reason of each session's satisfaction judgement. In all, a session-level evaluation model with improved performance and well capability on interpretation is proposed and applied in real practice in search engine companies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑