i-Razor:一种用于DNN推荐系统中特征选择与维度搜索的可微分神经输入剃刀
i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systems
AI总结:
针对DNN推荐系统中特征选择与维度搜索顺序优化导致次优配置的问题,提出可微分神经输入剃刀i-Razor,实现联合优化,并在大规模CTR数据集上验证其有效性与优越性。
摘要:
输入特征在基于DNN的推荐系统中扮演着至关重要的角色,这些系统包含来自用户、物品、上下文和交互的数千个类别和连续字段。噪声特征和不恰当的嵌入维度分配会降低推荐系统的性能,并在模型训练和在线服务中引入不必要的复杂性。优化DNN模型的输入配置,包括特征选择和嵌入维度分配,已成为特征工程中的关键主题之一。然而,在现有的工业实践中,特征选择和维度搜索是顺序优化的,即首先进行特征选择,然后进行维度搜索以确定每个选定特征的最佳维度大小。这种顺序优化机制增加了训练成本,并可能产生次优的输入配置。为解决这一问题,我们提出了一种可微分的神经输入剃刀(i-Razor),能够实现特征选择和维度搜索的联合优化。具体而言,我们引入了一个端到端的可微分模型来学习每个特征不同嵌入区域的相对重要性。此外,我们提出了一种灵活的剪枝算法,以同时实现特征过滤和维度推导。在点击率(CTR)预测任务中,对两个大规模公共数据集的广泛实验证明了i-Razor在平衡模型复杂性和性能方面的有效性和优越性。
英文摘要:
Input features play a crucial role in DNN-based recommender systems with thousands of categorical and continuous fields from users, items, contexts, and interactions. Noisy features and inappropriate embedding dimension assignments can deteriorate the performance of recommender systems and introduce unnecessary complexity in model training and online serving. Optimizing the input configuration of DNN models, including feature selection and embedding dimension assignment, has become one of the essential topics in feature engineering. However, in existing industrial practices, feature selection and dimension search are optimized sequentially, i.e., feature selection is performed first, followed by dimension search to determine the optimal dimension size for each selected feature. Such a sequential optimization mechanism increases training costs and risks generating suboptimal input configurations. To address this problem, we propose a differentiable neural input razor (i-Razor) that enables joint optimization of feature selection and dimension search. Concretely, we introduce an end-to-end differentiable model to learn the relative importance of different embedding regions of each feature. Furthermore, a flexible pruning algorithm is proposed to achieve feature filtering and dimension derivation simultaneously. Extensive experiments on two large-scale public datasets in the Click-Through-Rate (CTR) prediction task demonstrate the efficacy and superiority of i-Razor in balancing model complexity and performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。