Foundation models have revolutionized tasks in computer vision and natural language processing. However, in the realm of tabular data, tree-based models like XGBoost continue to dominate. TabPFN, a transformer model tailored for tabular data, mirrors recent foundation models in its exceptional in-context learning capability, being competitive with XGBoost's performance without the need for task-specific training or hyperparameter tuning. Despite its promise, TabPFN's applicability is hindered by its data size constraint, limiting its use in real-world scenarios. To address this, we present in-context data distillation (ICD), a novel methodology that effectively eliminates these constraints by optimizing TabPFN's context. ICD efficiently enables TabPFN to handle significantly larger datasets with a fixed memory budget, improving TabPFN's quadratic memory complexity but at the cost of a linear number of tuning steps. Notably, TabPFN, enhanced with ICD, demonstrates very strong performance against established tree-based models and modern deep learning methods on 48 large tabular datasets from OpenML.
Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data
Department of Statistics, Texas A\&M University, College Station, TX(德克萨斯A&M大学统计学系);Department of Electrical and Computer Engineering, University of Minnesota, Twin City, MN(明尼苏达大学电气与计算机工程系)
A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities
School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);The Ohio State University(俄亥俄州立大学)
Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data
University of Freiburg(弗莱堡大学);Technical University of Munich(慕尼黑工业大学);ETH Zurich(苏黎世联邦理工学院);Charité University Medicine Berlin(柏林夏里特医学院);ELLIS Institute Tübingen(ELLIS图宾根研究所)
Class-Imbalanced-Aware Adaptive Dataset Distillation for Scalable Pretrained Model on Credit Scoring
Risk Management Division, China Development Bank(国家开发银行风险管理部);School of Finance, Central University of Finance and Economics(中央财经大学金融学院);GF Holdings (Hong Kong) Corporation Limited(广发控股(香港)有限公司);University of Cambridge(剑桥大学)
Tabular foundation model for GEOAI benchmark problems BM/AirportSoilProperties/2/2025
Department of Civil and Environmental Engineering, Tohoku University(土屋大学土木环境工程系);Research Organization of Information and Systems, The Institute of Statistical Mathematics(信息与系统研究所,统计数学研究所);The Graduate University for Advanced Studies, Department of Statistical Science(高级研究大学,统计科学系)
Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN
University of Bergen Centre for the Science of Learning \& Technology (SLATE) Bergen Norway;Department of Computer Science University of Oxford;Alan Turing Institute Oxford United Kingdom;University of Bergen;University of Oxford;Alan Turing Institute
TabPFN Extensions for Interpretable Geotechnical Modelling
Department of Civil and Environmental Engineering, Tohoku University(东北大学土木环境工程系);Research Organization of Information and Systems, The Institute of Statistical Mathematics(信息与系统研究所统计数学研究所);Department of Statistical Science, The Graduate University for Advanced Studies(高级研究大学统计科学系)
State-Space Models for Tabular Prior-Data Fitted Networks
L3S Research Center, Leibniz University Hannover, Hannover, Germany(L3S研究所以及汉诺威莱布尼茨大学);University of Applied Sciences Rosenheim, Rosenheim, Germany(罗斯海姆应用科学大学);Technical University of Munich, Munich, Germany(慕尼黑技术大学)
Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction
Dep. of Electrical and Computer Engineering, McGill University(麦吉尔大学电气与计算机工程系);Mila – Québec AI Institute(魁北克AI研究所);Dep. of Mining and Materials Engineering, McGill University(麦吉尔大学采矿与材料工程系);Algoma Steel Inc.(阿尔戈马钢铁公司)
From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy
Osnabrück University(奥斯纳布吕克大学);Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB)(莱布尼茨农业工程与生物经济研究所);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)
In-Context Learning to Assess Built Environment Impacts on Perceived Neighborhood Walkability Among Mobility-impaired Older Adults
Future Health Technologies, Singapore-ETH Centre(未来健康技术中心,新加坡-ETH中心);Institute of Geriatrics and Active Ageing, Tan Tock Seng Hospital(陈笃生医院老年病学与 active 老龄化研究所);Geriatric Medicine, Khoo Teck Puat Hospital(邱德拔医院老年医学科);Department of Orthopedic Surgery, Woodlands Health Campus(伍德兰兹健康园区骨科)
TSPFN: A Temporal Tabular Foundation Model for Physiological Time Series Classification
Sorbonne Université(索邦大学);CNRS(法国国家科学研究中心);ISIR(巴黎索邦大学机器人与智能系统研究所);MLIA(机器学习与人工智能实验室);Inria(法国国家信息与自动化研究所);I3S(信息科学与系统实验室);Institut Universitaire de France (IUF)(法国大学研究院);INSA-Lyon(里昂国立应用科学学院);Université Claude Bernard Lyon 1(里昂第一大学);Inserm(法国国家健康与医学研究院);CREATIS(图像、信号与智能系统研究中心)
Modern Neural Networks for Small Tabular Datasets: The New Default for Field-Scale Digital Soil Mapping?
Osnabrück University, Joint Lab Artificial Intelligence and Data Science(奥尔登堡大学,人工智能与数据科学联合实验室);Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB)(莱比锡农业工程与生物经济研究所(ATB));German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI))
Foundation-Model Surrogates Enable Data-Efficient Active Learning for Materials Discovery
Department of Materials Science and Engineering, University of Illinois Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校材料科学与工程系);Department of Computer Science & Engineering, University of South Carolina(南卡罗来纳大学计算机科学与工程系);Zuoyue Honors College, Hangzhou Dianzhi University(杭州电子科技大学卓越荣誉学院);Department of Mechanical Engineering, University of South Carolina(南卡罗来纳大学机械工程系)