正则化目标编码在具有高基数特征的有监督机器学习中优于传统方法
Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features
摘要：
由于大多数机器学习（ML）算法都是为数值输入而设计的，因此有效地编码分类变量是数据分析中的一个关键方面。一个常见的问题是高基数特征，即具有大量水平的无序分类预测变量。我们研究了产生分类变量的数值表示的技术，这些变量可以在随后的ML应用中使用。我们专注于这些技术对后续算法的预测性能的影响，并-如果可能的话-获得何时使用哪种技术的最佳实践。我们进行了一个大规模的基准测试实验，在实验中，我们使用回归、二进制和多类分类设置的数据集，比较了不同的编码策略和五种ML算法（套索、随机森林、梯度提升、k近邻、支持向量机）。在我们的研究中，目标编码的正则化版本（即使用基于训练集中特征水平的目标预测作为新的数值特征）始终提供最佳结果。传统上广泛使用的编码在创建二进制指示符变量之前做出不合理的假设来将级别映射到整数（例如整数编码）或减少级别的数量（可能基于目标信息，例如叶编码）（独热或虚拟编码）在比较中不是有效的。
英文摘要：
Since most machine learning (ML) algorithms are designed for numerical inputs, efficiently encoding categorical variables is a crucial aspect in data analysis. A common problem are high cardinality features, i.e. unordered categorical predictor variables with a high number of levels. We study techniques that yield numeric representations of categorical variables which can then be used in subsequent ML applications. We focus on the impact of these techniques on a subsequent algorithm's predictive performance, and -- if possible -- derive best practices on when to use which technique. We conducted a large-scale benchmark experiment, where we compared different encoding strategies together with five ML algorithms (lasso, random forest, gradient boosting, k-nearest neighbors, support vector machine) using datasets from regression, binary- and multiclass- classification settings. In our study, regularized versions of target encoding (i.e. using target predictions based on the feature levels in the training set as a new numerical feature) consistently provided the best results. Traditionally widely used encodings that make unreasonable assumptions to map levels to integers (e.g. integer encoding) or to reduce the number of levels (possibly based on target information, e.g. leaf encoding) before creating binary indicator variables (one-hot or dummy encoding) were not as effective in comparison.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
