arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.02527cs.LG
收藏

TabPFN Unleashed:一种可扩展且有效的表格分类问题解决方案

TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems

State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

Si-Yang Liu, Han-Jia Ye

AI总结:

本文提出 Beta 方法,通过轻量级编码器微调和装袋策略同时降低 TabPFN 的偏差与方差,在 200 多个基准数据集上取得领先性能。

摘要:

TabPFN 已成为一种有前景的表格数据上下文学习模型,能够在给定带标签训练样本的情况下直接预测测试样本的标签。它已展现出具有竞争力的性能,尤其是在小规模分类任务上。然而,尽管 TabPFN 效果显著,它仍需要在若干方面进一步改进,包括处理高维特征、与下游数据集对齐以及扩展到更大数据集。在本文中,我们重新审视了 TabPFN 的现有变体,并观察到大多数方法要么侧重于降低偏差,要么侧重于降低方差,往往忽略了需要同时解决另一方面的问题,同时还增加了推理开销。为了填补这一空白,我们提出了 Beta(Bagging and Encoder-based Fine-tuning for TabPFN Adaptation,即面向 TabPFN 适应的装袋与基于编码器的微调方法),这是一种新颖且有效的方法,旨在同时最小化偏差和方差。为了降低偏差,我们引入了一个轻量级编码器,以更好地将下游任务与预训练的 TabPFN 对齐。通过以轻量级方式增加编码器数量,Beta 缓解了方差问题,从而进一步提升了模型性能。此外,还采用了自助采样(bootstrapped sampling)来进一步降低数据扰动对模型的影响,同时在推理过程中保持计算效率。我们的方法增强了 TabPFN 处理高维数据的能力,并使其能够扩展到更大的数据集。在 200 多个基准分类数据集上的实验结果表明,Beta 要么优于最先进的方法,要么与之持平。

英文摘要:

TabPFN has emerged as a promising in-context learning model for tabular data, capable of directly predicting the labels of test samples given labeled training examples. It has demonstrated competitive performance, particularly on small-scale classification tasks. However, despite its effectiveness, TabPFN still requires further refinement in several areas, including handling high-dimensional features, aligning with downstream datasets, and scaling to larger datasets. In this paper, we revisit existing variants of TabPFN and observe that most approaches focus either on reducing bias or variance, often neglecting the need to address the other side, while also increasing inference overhead. To fill this gap, we propose Beta (Bagging and Encoder-based Fine-tuning for TabPFN Adaptation), a novel and effective method designed to minimize both bias and variance. To reduce bias, we introduce a lightweight encoder to better align downstream tasks with the pre-trained TabPFN. By increasing the number of encoders in a lightweight manner, Beta mitigate variance, thereby further improving the model's performance. Additionally, bootstrapped sampling is employed to further reduce the impact of data perturbations on the model, all while maintaining computational efficiency during inference. Our approach enhances TabPFN's ability to handle high-dimensional data and scale to larger datasets. Experimental results on over 200 benchmark classification datasets demonstrate that Beta either outperforms or matches state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑