arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.01848cs.LGstat.ML
收藏

TabPFN:一种一秒内解决小规模表格分类问题的Transformer

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

University of Freiburg(弗莱堡大学);Charité University Medicine Berlin(柏林夏里特医学院);Bosch Center for Artificial Intelligence(博世人工智能中心)

Noah Hollmann, Samuel Müller, Katharina Eggensperger, Frank Hutter

AI总结:

TabPFN通过先验数据拟合网络和上下文学习,在无需超参数调优的情况下,以显著的速度优势实现与最先进AutoML系统相当的小规模表格分类性能。

摘要:

我们提出了TabPFN,一种经过训练的Transformer,它可以在不到一秒的时间内对小规模表格数据集进行监督分类,无需超参数调优,并且与最先进的分类方法具有竞争力。TabPFN执行上下文学习(ICL),它学习利用输入中给出的带标签样本序列(x, f(x))进行预测,而无需进一步的参数更新。TabPFN完全蕴含在我们网络的权重中,该网络接受训练样本和测试样本作为集合值输入,并在单次前向传播中产生整个测试集的预测。TabPFN是一种先验数据拟合网络(PFN),离线训练一次,以在从我们的先验中抽取的合成数据集上近似贝叶斯推理。该先验融入了因果推理的思想:它涵盖了一个大的结构因果模型空间,并偏好简单结构。在OpenML-CC18套件中满足以下条件的18个数据集上——最多1000个训练数据点、最多100个无缺失值的纯数值特征以及最多10个类别——我们表明,我们的方法明显优于提升树,并且与复杂的最先进AutoML系统性能相当,同时实现了高达230倍的速度提升。当使用GPU时,这一速度提升增加到5700倍。我们还在OpenML中另外67个小型数值数据集上验证了这些结果。我们在https://github.com/automl/TabPFN提供了所有代码、训练好的TabPFN、一个交互式浏览器演示以及一个Colab笔记本。

英文摘要:

We present TabPFN, a trained Transformer that can do supervised classification for small tabular datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods. TabPFN performs in-context learning (ICL), it learns to make predictions using sequences of labeled examples (x, f(x)) given in the input, without requiring further parameter updates. TabPFN is fully entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. This prior incorporates ideas from causal reasoning: It entails a large space of structural causal models with a preference for simple structures. On the 18 datasets in the OpenML-CC18 suite that contain up to 1 000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes, we show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 230$\times$ speedup. This increases to a 5 700$\times$ speedup when using a GPU. We also validate these results on an additional 67 small numerical datasets from OpenML. We provide all our code, the trained TabPFN, an interactive browser demo and a Colab notebook at https://github.com/automl/TabPFN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑