arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.00430cs.AIcs.IR
收藏

利用缩放定律和近似熵优化序列推荐模型

Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy

Huawei Noah’s Ark Lab(华为诺亚方舟实验室);University of Science and Technology of China(中国科学技术大学)

Tingjia Shen, Hao Wang, Chuhan Wu, Jin Yao Chin, Wei Guo, Yong Liu, Huifeng Guo, Defu Lian, Ruiming Tang, Enhong Chen

AI总结:

针对缩放定律难以直接用于序列推荐的问题,提出基于HR、NDCG拟合和近似熵衡量数据质量的性能定律,以预测不同数据与模型规模下的性能。

摘要:

缩放定律已成为理解模型性能如何随规模增大而演变的强大框架,为优化计算资源提供了宝贵洞见。在序列推荐(SR)领域——该任务对预测用户的序列偏好至关重要——这些定律提供了一种视角,可用于应对SR模型可扩展性带来的挑战。然而,推荐系统中存在的结构性和协同性问题,阻碍了缩放定律(SL)在这些系统中的直接应用。为此,我们提出了面向SR模型的性能定律,旨在从理论上研究并建模模型性能与数据质量之间的关系。具体而言,我们首先将HR和NDCG指标拟合到基于transformer的SR模型。随后,我们提出使用近似熵(ApEn)来评估数据质量,与传统的数据数量指标相比,该方法提供了更细致的分析途径。我们的方法能够在不同数据集规模和模型大小下实现准确预测,在大型SR模型中展现出强相关性,并为在任意给定模型配置下获得最优性能提供了洞见。

英文摘要:

Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑