TabFM-Auto：表格基础模型的自进化流水线

中文摘要 AI 辅助

表格基础模型通过在合成表格上进行预训练，在结构化数据上实现了强大的零样本准确率，但它们忽略了承载数据集语义的列名、任务描述和辅助文件。与此同时，自进化机器学习工程（MLE）智能体在每个数据集上从头训练模型，但联合搜索特征、架构和超参数是嘈杂的且容易过拟合。我们提出了TabFM-Auto，它将表格基础模型TabFM与一个语言模型智能体配对，该智能体围绕TabFM进化数据流水线。在数据集元数据和验证反馈的引导下，TabFM-Auto迭代地优化数据清洗、特征工程、上下文选择和后处理，以减少TabFM的错误。在TabArena基准的所有51个数据集上，五个使用不同智能体和语言模型的TabFM-Auto配置占据了总体前五名，最佳配置将TabFM的Elo评分从1785提升到2013。发现的流水线也能迁移到其他冻结的表格基础模型（+69到+143 Elo），无需进一步搜索。在MLE-Bench的8个表格竞赛中，TabFM-Auto在MLE智能体中总体排名第一。

英文摘要

Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.

发表机构

• Google Research（谷歌研究院）
• University of Southern California（南加州大学）
• Google DeepMind（谷歌DeepMind）
• Harvard University（哈佛大学）
• University of Texas at Austin（德克萨斯大学奥斯汀分校）

机构由 AI 辅助整理，请以论文原文为准。

相关深度报道

• Google让Agent自己改数据流水线，51个表格任务直接包揽前五