arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.06809cs.IRcs.AI
收藏

生成多样化合成数据集以评估现实生活中的推荐系统

Generating Diverse Synthetic Datasets for Evaluation of Real-life Recommender Systems

Faculty of Computer and Information Science(计算机与信息科学学院)

Miha Malenšek, Blaž Škrlj, Blaž Mramor, Jure Demšar

AI总结:

本文提出一个模块化开源 Python 框架,用于生成统计一致、可定制的高维稀疏合成类别数据集,支持推荐系统的算法基准测试、偏见检测与 AutoML 搜索评估。

摘要:

合成数据集对于评估和测试机器学习模型非常重要。在评估现实生活中的推荐系统时,通常会考虑高维类别型(且稀疏的)数据集。遗憾的是,能够生成具有此类特征的人工数据集的解决方案并不多。为此,我们开发了一个新颖的框架,用于生成多样化且统计一致的合成数据集。我们的框架支持创建具有受控属性的数据集,并允许进行迭代修改以满足特定实验需求,例如引入复杂的特征交互、特征基数或特定分布。我们通过概率计数算法基准测试、算法偏见检测以及模拟 AutoML 搜索等用例展示了该框架的实用性。现有方法要么狭隘地聚焦于特定数据集结构,要么优先通过真实数据进行(私有)数据合成;与之不同,我们的方法提供了一种模块化方式,可快速生成完全合成的数据集,并能根据多样化实验需求进行定制。我们的结果表明,该框架能够在独特情形中有效隔离模型行为,并凸显其在推荐系统评估与开发方面取得重大进展的潜力。这个现成可用的框架以免费开源 Python 包形式提供,以便以最小阻力促进研究。

英文摘要:

Synthetic datasets are important for evaluating and testing machine learning models. When evaluating real-life recommender systems, high-dimensional categorical (and sparse) datasets are often considered. Unfortunately, there are not many solutions that would allow generation of artificial datasets with such characteristics. For that purpose, we developed a novel framework for generating synthetic datasets that are diverse and statistically coherent. Our framework allows for creation of datasets with controlled attributes, enabling iterative modifications to fit specific experimental needs, such as introducing complex feature interactions, feature cardinality, or specific distributions. We demonstrate the framework's utility through use cases such as benchmarking probabilistic counting algorithms, detecting algorithmic bias, and simulating AutoML searches. Unlike existing methods that either focus narrowly on specific dataset structures, or prioritize (private) data synthesis through real data, our approach provides a modular means to quickly generating completely synthetic datasets we can tailor to diverse experimental requirements. Our results show that the framework effectively isolates model behavior in unique situations and highlights its potential for significant advancements in the evaluation and development of recommender systems. The readily-available framework is available as a free open Python package to facilitate research with minimal friction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑