AutoFT：点击率预测中参数迁移学习的自动微调
AutoFT: Automatic Fine-Tune for Parameters Transfer Learning in Click-Through Rate Prediction
摘要：
推荐系统经常被要求服务于多个推荐场景或领域。从源域微调预训练的CTR模型并使其适应目标域允许知识转移。然而，如果目标数据集很小而参数的数量很大，那么优化预训练网络的所有参数可能会导致过度拟合。这让我们想到直接重用预训练模型中的参数，这些参数代表了从多个领域学习到的更一般的特征。然而，冻结或微调参数层的设计需要大量的手动工作，因为决策高度依赖于预先训练的模型和目标实例。在这项工作中，我们提出了一个端到端的迁移学习框架，称为自动微调（AutoFT），CTR预测。AutoFT由逐域迁移策略和逐层迁移策略组成。逐字段传输策略决定如何基于来自目标域的给定实例冻结或微调预训练的嵌入表示。分层转移政策决定了高？顺序特征表示被逐层传递。在两个公共基准数据集和一个私有工业数据集上的大量实验表明，与最先进的传输方法相比，AutoFT可以显着提高CTR预测的性能。
英文摘要：
Recommender systems are often asked to serve multiple recommendation scenarios or domains. Fine-tuning a pre-trained CTR model from source domains and adapting it to a target domain allows knowledge transferring. However, optimizing all the parameters of the pre-trained network may result in over-fitting if the target dataset is small and the number of parameters is large. This leads us to think of directly reusing parameters in the pre-trained model which represent more general features learned from multiple domains. However, the design of freezing or fine-tuning layers of parameters requires much manual effort since the decision highly depends on the pre-trained model and target instances. In this work, we propose an end-to-end transfer learning framework, called Automatic Fine-Tuning (AutoFT), for CTR prediction. AutoFT consists of a field-wise transfer policy and a layer-wise transfer policy. The field-wise transfer policy decides how the pre-trained embedding representations are frozen or fine-tuned based on the given instance from the target domain. The layer-wise transfer policy decides how the high?order feature representations are transferred layer by layer. Extensive experiments on two public benchmark datasets and one private industrial dataset demonstrate that AutoFT can significantly improve the performance of CTR prediction compared with state-of-the-art transferring approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
