Eval4DiRec：基于扩散的推荐系统的统一与系统性评估框架
Eval4DiRec: A Unified and Systematic Evaluation Framework for Diffusion-based Recommender Systems
浏览论文内容
中文总结 AI 辅助
针对基于扩散的推荐系统缺乏统一评估基准的问题，提出Eval4DiRec框架，支持14个模型、5种场景，通过实证研究揭示关键因素，促进公平评估与未来研究。
中文摘要 AI 辅助
利用扩散模型强大的生成能力和稳定的训练动态，基于扩散的推荐系统（RSs）近年来已成为一种新颖的推荐范式，吸引了学术界和工业界越来越多的关注。然而，尽管基于扩散的推荐系统发展迅速，一个关键问题已经出现：缺乏统一且系统性的定量评估基准，这常常导致由于数据处理、训练配置、推理过程和评估协议的不一致，实验结果无法复现，且研究之间的比较不公平。为了应对这一挑战，我们提出了Eval4DiRec，这是第一个专门为基于扩散的推荐系统设计的统一且开源的评估框架。Eval4DiRec支持五种不同推荐场景下的14个具有代表性的基于扩散的推荐系统模型，提供一致且可复现的实验设置，以系统性地评估其性能。基于该框架，我们进行了广泛的实证研究，在统一协议下对这些模型进行基准测试。结果凸显了扩散模型在推荐方面的强大潜力，同时也揭示了显著影响其性能的关键因素和实际挑战，从而为促进公平评估和指导这一有前景领域的未来研究奠定了坚实基础。我们的代码和数据可在以下网址获取：此https URL。
英文摘要
Leveraging the strong generative capabilities and stable training dynamics of diffusion models, diffusion-based recommender systems (RSs) have recently emerged as a novel recommendation paradigm, attracting increasing attention from both academia and industry. However, despite the rapid growth of diffusion-based RSs, a critical issue has emerged: the lack of a unified and systematic quantitative evaluation benchmark, which often results in irreproducible experimental results and unfair comparisons across studies due to inconsistent data processing, training configurations, inference procedures, and evaluation protocols. To address this challenge, we propose Eval4DiRec, the first unified and open-source evaluation framework specifically designed for diffusion-based RSs. Eval4DiRec supports 14 representative diffusion-based RS models across five different recommendation scenarios, providing consistent and reproducible experimental settings to systematically assess their performance. Built upon this framework, we conduct extensive empirical studies to benchmark these models under unified protocols. The results highlight the strong potential of diffusion models for recommendation while also revealing key factors and practical challenges that substantially affect their performance, thereby establishing a solid foundation to facilitate fair evaluation and guide future research in this promising field. Our code and data are available at: https://github.com/wangcong2001/Eval4DiRec.
发表机构
- Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））
- Shandong Computer Science Center (National Supercomputer Center in Jinan)（山东省计算中心（国家超级计算济南中心））
- University of Technology Sydney（悉尼科技大学）
- Tongji University（同济大学）
- Shandong Academy of Artificial Intelligence（山东省人工智能研究院）
机构由 AI 辅助整理，请以论文原文为准。
