arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.13952cs.IRcs.AI
收藏

知识蒸馏方法用于准确且高效的推荐系统

Knowledge Distillation Approaches for Accurate and Efficient Recommender System

Pohang University of Science and Technology(浦项科技大学)

SeongKu Kang

AI总结:

本论文提出多种知识蒸馏方法,包括潜在知识与排序知识迁移,以提升紧凑推荐模型的性能,实现更好的精度-效率权衡。

摘要:

尽管知识蒸馏(KD)在分类问题中取得了突破,但在以往文献中,将其应用于推荐模型和排序问题的研究尚不充分。本论文致力于开发用于推荐系统的知识蒸馏方法,以全面提升紧凑模型的性能。我们提出了专为推荐系统设计的新型蒸馏方法。所提出的方法根据其知识来源分类如下:(1)潜在知识:我们提出了两种方法,用于迁移用户/物品表示的潜在知识。它们通过一种平衡的蒸馏策略有效迁移小众品味的知识,该策略防止KD过程偏向于少数大型偏好群体。此外,我们提出了一种新方法,用于迁移表示空间中的用户/物品关系。所提出的方法考虑到紧凑模型的有限容量,选择性地迁移关键关系。(2)排序知识:我们提出了三种方法,用于从推荐结果中迁移排序知识。它们将KD过程表述为排序匹配问题,并通过列表式学习策略迁移知识。进一步,我们提出了一种新的学习框架,用于压缩异构推荐模型的排序知识。该框架旨在减轻模型集成的计算负担,而模型集成是许多推荐应用中的主导解决方案。我们通过大量实验验证了所提出方法和框架的益处。总之,本论文揭示了知识蒸馏方法在实现推荐模型更好的精度-效率权衡方面的作用。

英文摘要:

Despite its breakthrough in classification problems, Knowledge distillation (KD) to recommendation models and ranking problems has not been studied well in the previous literature. This dissertation is devoted to developing knowledge distillation methods for recommender systems to fully improve the performance of a compact model. We propose novel distillation methods designed for recommender systems. The proposed methods are categorized according to their knowledge sources as follows: (1) Latent knowledge: we propose two methods that transfer latent knowledge of user/item representation. They effectively transfer knowledge of niche tastes with a balanced distillation strategy that prevents the KD process from being biased towards a small number of large preference groups. Also, we propose a new method that transfers user/item relations in the representation space. The proposed method selectively transfers essential relations considering the limited capacity of the compact model. (2) Ranking knowledge: we propose three methods that transfer ranking knowledge from the recommendation results. They formulate the KD process as a ranking matching problem and transfer the knowledge via a listwise learning strategy. Further, we present a new learning framework that compresses the ranking knowledge of heterogeneous recommendation models. The proposed framework is developed to ease the computational burdens of model ensemble which is a dominant solution for many recommendation applications. We validate the benefit of our proposed methods and frameworks through extensive experiments. To summarize, this dissertation sheds light on knowledge distillation approaches for a better accuracy-efficiency trade-off of the recommendation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑