重访表格数据的最近邻：二十年后的深度表格基线
Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later
AI总结：
本文重访经典方法邻域成分分析（NCA），通过结合SGD、深度表示和训练随机性等现代技术，使其在300个数据集上性能媲美CatBoost，优于现有深度表格模型。
摘要：
对深度学习的广泛热情最近扩展到了表格数据领域。认识到深度表格方法的进步往往受到经典方法的启发，例如将最近邻整合到神经网络中，我们研究这些经典方法能否通过现代技术焕发新生。我们重新审视了 $K$-最近邻（KNN）的可微版本——邻域成分分析（NCA），它最初被设计用于学习线性投影以捕捉实例间的语义相似性，并尝试在此基础上逐步添加现代深度学习技术。令人惊讶的是，我们使用 SGD 实现且不进行降维的 NCA 在表格数据上已经取得了不错的性能，这与使用 scikit-learn 等现有工具箱的结果形成对比。进一步为 NCA 配备深度表示和额外的训练随机性显著增强了其能力，在 300 个数据集的分类和回归任务中，其表现与领先的基于树的方法 CatBoost 持平，并优于现有的深度表格模型。我们最后分析了这些改进背后的因素，包括损失函数、预测策略和深度架构。代码可在 https://github.com/qile2000/LAMDA-TALENT 获取。
英文摘要：
The widespread enthusiasm for deep learning has recently expanded into the domain of tabular data. Recognizing that the advancement in deep tabular methods is often inspired by classical methods, e.g., integration of nearest neighbors into neural networks, we investigate whether these classical methods can be revitalized with modern techniques. We revisit a differentiable version of $K$-nearest neighbors (KNN) -- Neighbourhood Components Analysis (NCA) -- originally designed to learn a linear projection to capture semantic similarities between instances, and seek to gradually add modern deep learning techniques on top. Surprisingly, our implementation of NCA using SGD and without dimensionality reduction already achieves decent performance on tabular data, in contrast to the results of using existing toolboxes like scikit-learn. Further equipping NCA with deep representations and additional training stochasticity significantly enhances its capability, being on par with the leading tree-based method CatBoost and outperforming existing deep tabular models in both classification and regression tasks on 300 datasets. We conclude our paper by analyzing the factors behind these improvements, including loss functions, prediction strategies, and deep architectures. The code is available at https://github.com/qile2000/LAMDA-TALENT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
