arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.13202cs.IR
收藏

基于STELLA框架的在线信息检索评价

Online Information Retrieval Evaluation using the STELLA Framework

Timo Breuer, Narges Tavakolpoursaleh, Johann Schaible, Daniel Hienert, Philipp Schaer, Leyla Jael Castro

摘要:

让用户参与软件开发的早期阶段已经成为一种常见的策略,因为它使开发人员能够从一开始就考虑用户的需求。一旦一个系统投入生产,随着更多的信息可用,观察、评估和向用户学习的新机会就会出现。从用户那里收集信息来持续评估他们的行为是商业软件的常见做法,而克兰菲尔德范式仍然是学术界信息检索和推荐系统的首选。在这里,我们介绍了生活实验室STELLA项目,其目的是创建一个评估基础设施,允许实验系统与真实用户一起运行生产基于Web的学术搜索系统。STELLA结合了用户交互和日志文件分析,为学术搜索提供了大规模的A/B实验。

英文摘要:

Involving users in early phases of software development has become a common strategy as it enables developers to consider user needs from the beginning. Once a system is in production, new opportunities to observe, evaluate and learn from users emerge as more information becomes available. Gathering information from users to continuously evaluate their behavior is a common practice for commercial software, while the Cranfield paradigm remains the preferred option for Information Retrieval (IR) and recommendation systems in the academic world. Here we introduce the Infrastructures for Living Labs STELLA project which aims to create an evaluation infrastructure allowing experimental systems to run along production web-based academic search systems with real users. STELLA combines user interactions and log files analyses to enable large-scale A/B experiments for academic search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑