TabPFN has emerged as a promising in-context learning model for tabular data, capable of directly predicting the labels of test samples given labeled training examples. It has demonstrated competitive performance, particularly on small-scale classification tasks. However, despite its effectiveness, TabPFN still requires further refinement in several areas, including handling high-dimensional features, aligning with downstream datasets, and scaling to larger datasets. In this paper, we revisit existing variants of TabPFN and observe that most approaches focus either on reducing bias or variance, often neglecting the need to address the other side, while also increasing inference overhead. To fill this gap, we propose Beta (Bagging and Encoder-based Fine-tuning for TabPFN Adaptation), a novel and effective method designed to minimize both bias and variance. To reduce bias, we introduce a lightweight encoder to better align downstream tasks with the pre-trained TabPFN. By increasing the number of encoders in a lightweight manner, Beta mitigate variance, thereby further improving the model's performance. Additionally, bootstrapped sampling is employed to further reduce the impact of data perturbations on the model, all while maintaining computational efficiency during inference. Our approach enhances TabPFN's ability to handle high-dimensional data and scale to larger datasets. Experimental results on over 200 benchmark classification datasets demonstrate that Beta either outperforms or matches state-of-the-art methods.
Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data
Department of Statistics, Texas A\&M University, College Station, TX(德克萨斯A&M大学统计学系);Department of Electrical and Computer Engineering, University of Minnesota, Twin City, MN(明尼苏达大学电气与计算机工程系)
A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities
School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);The Ohio State University(俄亥俄州立大学)
Realistic Evaluation of TabPFN v2 in Open Environments
School of Intelligence Science and Technology, Nanjing University, China(智能科学与技术学院,南京大学);National Key Laboratory for Novel Software Technology, Nanjing University, China(新型软件技术国家实验室,南京大学);School of Artificial Intelligence, Nanjing University, China(人工智能学院,南京大学)
Tabular foundation model for GEOAI benchmark problems BM/AirportSoilProperties/2/2025
Department of Civil and Environmental Engineering, Tohoku University(土屋大学土木环境工程系);Research Organization of Information and Systems, The Institute of Statistical Mathematics(信息与系统研究所,统计数学研究所);The Graduate University for Advanced Studies, Department of Statistical Science(高级研究大学,统计科学系)
Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure
Department of Mathematics, Informatics and Geosciences(数学、信息学与地质科学系);University of Trieste(特里este大学);Department of Engineering and Architecture(工程与建筑系);InSilicoTrials Technologies BV(InSilicoTrials技术公司);Amsterdam UMC Institute for Logic, Language and Computation(阿姆斯特丹大学医学中心逻辑、语言与计算研究所);University of Amsterdam(阿姆斯特丹大学)
State-Space Models for Tabular Prior-Data Fitted Networks
L3S Research Center, Leibniz University Hannover, Hannover, Germany(L3S研究所以及汉诺威莱布尼茨大学);University of Applied Sciences Rosenheim, Rosenheim, Germany(罗斯海姆应用科学大学);Technical University of Munich, Munich, Germany(慕尼黑技术大学)
Longitudinal Progression Prediction of Alzheimer's Disease with Tabular Foundation Model
Department of Computer Science Emory University Atlanta GA USA(埃默里大学计算机科学系);Center for Data Science Nell Hodgson Woodruff School of Nursing Emory University Atlanta GA USA(埃默里大学数据科学中心);Department of Computer Science Georgia State University Atlanta GA USA(佐治亚州立大学计算机科学系);Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) Atlanta GA USA(神经影像与数据科学转化研究三机构中心(TReNDS))
From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy
Osnabrück University(奥斯纳布吕克大学);Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB)(莱布尼茨农业工程与生物经济研究所);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)
On the Uncertainty Quantification Ability of Tabular Foundation Models
Department of Mechanical and Aerospace Engineering, University of California, Irvine(加州大学欧文分校机械与航空航天工程系);Department of Civil and Environmental Engineering, University of California, Irvine(加州大学欧文分校土木与环境工程系)
Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction
Dep. of Electrical and Computer Engineering, McGill University(麦吉尔大学电气与计算机工程系);Mila – Québec AI Institute(魁北克AI研究所);Dep. of Mining and Materials Engineering, McGill University(麦吉尔大学采矿与材料工程系);Algoma Steel Inc.(阿尔戈马钢铁公司)
Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data
University of Freiburg(弗莱堡大学);Technical University of Munich(慕尼黑工业大学);ETH Zurich(苏黎世联邦理工学院);Charité University Medicine Berlin(柏林夏里特医学院);ELLIS Institute Tübingen(ELLIS图宾根研究所)
Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN
University of Bergen Centre for the Science of Learning \& Technology (SLATE) Bergen Norway;Department of Computer Science University of Oxford;Alan Turing Institute Oxford United Kingdom;University of Bergen;University of Oxford;Alan Turing Institute
TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts
Institute for Bioinformatics;Medical Informatics , University of T\" u bingen , Maria-von-Linden-Str. 6 , 72076 , Baden-W\" u rttemberg , Germany;AutoML for Science , University of T\" u bingen , Maria-von-Linden-Str. 6 , 72076 , Baden-W\" u rttemberg , Germany;Lamarr Institute for Machine Learning
Qiong Zhang is Assistant Professor, Institute of Statistics \& Big Data, Renmin University of China, China e1 .;Yan Shuo Tan is Assistant Professor, Department of Statistics \& Data Science, National University of Singapore, Singapore e2 .;Qinglong Tian is Assistant Professor, Department of Statistics \& Actuarial Science, University or Waterloo, Canada e3 .;Pengfei Li is Professor, Department of Statistics \& Actuarial Science, University or Waterloo, Canada e4 .