arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.17065cs.LG
收藏

TILE:连接大语言模型(LLMs)与表格基础模型(TFMs)以实现基于字符串的表格学习

TILE: Bridging LLMs and Table Foundation Models for Tabular Learning with Strings

Inria Saclay(法国国家信息与自动化研究所萨克雷中心);SAP SE(思爱普公司)

Félix Lefebvre, David Holzmüller, Gioia Blayer, Myung Jun Kim, Marine Le Morvan, Gaël Varoquaux

AI总结:

本研究提出TILE接口,解决LLMs与TFMs衔接时的表格表征提取问题,结合行序列化等四个组件,在100个数据集等基准上实现含字符串表格学习的最优性能。

摘要:

现实世界的表格通常是异构的,数值特征与自由格式的字符串列并存。表格基础模型(TFMs)在数值表格上表现出色,但需要将字符串编码为数值;而大语言模型(LLMs)具备语义知识,却并非为表格预测设计。多数现有衔接方法将LLMs作为单元格字符串的插件编码器,但如何为表格学习有效提取表征仍未得到充分探索。我们发现LLMs与表格间的接口和LLM本身同样重要:按单元格应用时,更大的LLMs不会提供更好的特征;而使用我们的接口时,相同的LLMs会随规模提升持续改进。TILE(Table-Interfaced LLM Embeddings,表格接口型LLM嵌入)包含四个关键组件:行序列化以提供跨列上下文、中间层提取以避免专为下一个token预测优化的层、列池化以消除均值池化的长度偏差,以及旁路特征以将数值和类别信息绕过LLM传递。与TFMs结合后,TILE在含字符串的表格任务上达到了新的最优性能。在100个数据集上,它在89%的划分上击败了最强基线,在7种LLMs和10种表格学习器上均有提升,且在包含108个数据集的STRABLE基准中排名第一。

英文摘要:

Real-world tables are often heterogeneous, combining numerical features with free-form string columns. Table foundation models (TFMs) excel on numerical tables but require strings to be encoded numerically, while large language models (LLMs) hold semantic knowledge but are not designed for tabular predictions. Most existing bridges use LLMs as plug-in encoders for cell strings, yet how to extract representations effectively for tabular learning remains underexplored. We show that the interface between LLMs and tables matters as much as the LLM itself. Applied cell-wise, larger LLMs do not give better features. With our interface, the same LLMs improve steadily with size. TILE (Table-Interfaced LLM Embeddings) combines four ingredients: row serialization to provide cross-column context, intermediate-layer extraction to avoid layers specialized for next-token prediction, columnar pooling to remove mean pooling's length bias, and by-pass features to route numbers and categories past the LLM. Paired with TFMs, TILE sets a new state of the art for tables with strings. Across 100 datasets, it beats the strongest baseline on $90\%$ of splits, with gains across 7 LLMs and 10 tabular learners, and ranks first on STRABLE's 108 datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑