arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.09320cs.IR
收藏

推荐系统中的群组验证:多层次性能评估框架

Group Validation in Recommender Systems: Framework for Multi-layer Performance Evaluation

Wissam Al Jurdi, Jacques Bou Abdo, Jacques Demerjian, Abdallah Makhoul

摘要:

解释试图在使用人工智能的平台上实现用户行为的模型的性能结果是研究人员和实践者继续面临的一个巨大挑战。虽然目前的评价工具有能力对一个系统的业绩提供可靠的总体概述,但它们在使用方面仍然缺乏一致性和有效性,这一点在最近关于这一专题的研究中显而易见。目前传统的评估技术往往无法检测到可能发生在较小的数据子集上的变化,并且缺乏解释这些变化如何影响整体性能的能力。在这篇文章中,我们专注于数据聚类的概念,用于评估在推荐系统应用程序的数据集,并应用邻域评估方法。这种新方法被称为基于邻域的评估,有助于更好地理解系统中更紧凑子集的关键性能变化,以帮助发现传统指标通常未注意到的弱点,并且通常被平均化。这个新的模块化评估层补充了现有的评估机制,并为推荐生态系统提供了几种应用的可能性,例如模型进化测试,欺诈/攻击检测以及托管混合模型设置的可能性。

英文摘要:

Interpreting the performance results of models that attempt to realize user behavior in platforms that employ recommenders is a big challenge that researchers and practitioners continue to face. Although current evaluation tools possess the capacity to provide solid general overview of a system's performance, they still lack consistency and effectiveness in their use as evident in most recent studies on the topic. Current traditional assessment techniques tend to fail to detect variations that could occur on smaller subsets of the data and lack the ability to explain how such variations affect the overall performance. In this article, we focus on the concept of data clustering for evaluation in recommenders and apply a neighborhood assessment method for the datasets of recommender system applications. This new method, named neighborhood-based evaluation, aids in better understanding critical performance variations in more compact subsets of the system to help spot weaknesses where such variations generally go unnoticed with conventional metrics and are typically averaged out. This new modular evaluation layer complements the existing assessment mechanisms and provides the possibility of several applications to the recommender ecosystem such as model evolution tests, fraud/attack detection and a possibility for hosting a hybrid model setup.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑