面向CTR预测的特征精炼模块的全面总结与评估
A Comprehensive Summarization and Evaluation of Feature Refinement Modules for CTR Prediction
AI总结：
本文系统定义并评估可插入嵌入层与交互层之间的特征精炼（FR）模块，基于14个模块、200多个增强模型和4000多次运行给出选型指导，并提出面向并行CTR模型的独立FR架构。
摘要：
点击率（CTR）预测在学术界和工业界都有广泛应用。大多数CTR任务都遵循特征嵌入与特征交互范式，其中CTR预测的准确率主要通过设计实用的特征交互结构来提升。然而，近期研究认为，仅通过嵌入层学习到的固定特征嵌入限制了现有CTR模型的性能。一些工作在嵌入层之上应用额外模块，以在不同实例中动态精炼特征表示，这使其有效且易于与现有CTR方法集成。尽管取得了有前景的结果，但在CTR任务上仍缺乏对这一新兴且有前景方向的系统综述和总结。为填补这一空白，我们全面总结并定义了一种可应用于特征嵌入层与交互层之间的新模块，即特征精炼（FR）模块。我们从以往工作中提取了14个FR模块，包括那些提出了FR模块但未明确定义或解释的情形。我们通过全面且大量的实验，充分评估了现有FR模块的有效性和兼容性；这些实验涉及200多个增强模型、4000多次运行以及超过15000个GPU小时。实验结果为研究人员提供了富有洞察力的指导方针，并且所有基准测试代码和实验结果均已开源。此外，我们提出了一种新架构，为并行CTR模型的各个独立子网络分配独立的FR模块，而不是采用在嵌入层之上插入共享FR模块的传统方法。我们的方法同样得到了全面实验的支持，实验证明了其有效性。
英文摘要：
Click-through rate (CTR) prediction is widely used in academia and industry. Most CTR tasks fall into a feature embedding \& feature interaction paradigm, where the accuracy of CTR prediction is mainly improved by designing practical feature interaction structures. However, recent studies have argued that the fixed feature embedding learned only through the embedding layer limits the performance of existing CTR models. Some works apply extra modules on top of the embedding layer to dynamically refine feature representations in different instances, making it effective and easy to integrate with existing CTR methods. Despite the promising results, there is a lack of a systematic review and summarization of this new promising direction on the CTR task. To fill this gap, we comprehensively summarize and define a new module, namely \textbf{feature refinement} (FR) module, that can be applied between feature embedding and interaction layers. We extract 14 FR modules from previous works, including instances where the FR module was proposed but not clearly defined or explained. We fully assess the effectiveness and compatibility of existing FR modules through comprehensive and extensive experiments with over 200 augmented models and over 4,000 runs for more than 15,000 GPU hours. The results offer insightful guidelines for researchers, and all benchmarking code and experimental results are open-sourced. In addition, we present a new architecture of assigning independent FR modules to separate sub-networks for parallel CTR models, as opposed to the conventional method of inserting a shared FR module on top of the embedding layer. Our approach is also supported by comprehensive experiments demonstrating its effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
