arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06290cs.AI
收藏

数据语言模型:一种新的用于表格数据的基础模型类别

Data Language Models: A New Foundation Model Class for Tabular Data

SchemaLabs

Eda Erol, Giuliano Pezzoli, Ozer Cem Kelahmet

AI总结:

本文提出数据语言模型(DLM),一种能直接理解表格数据的基础模型,无需预处理,提升了表格数据在AI决策中的应用效率。

摘要:

如今,每种主要数据模态都有对应的基础模型:文本有语言模型,图像有视觉模型,音频有音频模型。然而,表格数据,许多现实世界AI决策所依赖的模态,却没有。目前所有表格AI方法,从梯度提升树到最新表格基础模型,都需要预处理管道才能使用数据。我们引入数据语言模型(DLM),即缺失的表格数据基础模型。DLM理解表格的方式如同语言模型理解句子:原生地,无需序列化或预处理,直接从原始单元格值中理解。它为AI模型、代理和垂直AI应用提供了表格数据层,消除了当前存在于原始数据和每种AI系统之间的预处理管道。我们展示了Schema-1,第一个DLM:一个1.4亿参数模型,训练于超过230万个合成和现实世界表格数据集。Schema-1在行级预测基准上优于梯度提升集成、AutoML堆栈和我们评估的表格基础模型。在缺失值重建方面,它在平均性能条件下比所有经典统计方法和前沿大语言模型的重建误差更低,证明了对数据集自身分布几何结构的理解比语言中编码的世界知识更适用于插补。它能够仅凭原始单元格值识别任何未见过的数据集的行业领域,可靠地在任何领域执行,这是先前所有表格模型都无法做到的。它是在AI堆栈中缺失的原生表格理解层。

英文摘要:

Every major data modality now has a foundation model that understands it natively: text has language models, images have vision models, audio has audio models. Tabular data, the modality on which many consequential real-world AI decisions are made, does not. Every approach to tabular AI today, from gradient-boosted trees to the latest tabular foundation models, requires a preprocessing pipeline before any model can consume the data. None of them understand tabular data as a modality. We introduce the Data Language Model (DLM), the missing foundation model for tabular data. A DLM understands tables the way a language model understands sentences: natively, without serialization or preprocessing, directly from raw cell values. It is the tabular data layer on which AI models, agents, and vertical AI applications can be built, eliminating the preprocessing pipelines that currently stand between raw data and every AI system that consumes it. We present Schema-1, the first DLM: a 140M parameter model trained on more than 2.3M synthetic and real-world tabular datasets. Schema-1 outperforms gradient-boosted ensembles, AutoML stacks, and the tabular foundation models we evaluate on established row-level prediction benchmarks. On missing value reconstruction it achieves lower reconstruction error than all classical statistical methods and frontier large language models on mean performance across conditions, establishing that structural understanding of a dataset's own distributional geometry is more useful for imputation than world knowledge encoded in language. It identifies the industry sector of any unseen dataset from raw cell values alone, reliably across any domain, a task no prior tabular model can perform. It is the native tabular understanding layer that has been missing from the AI stack.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑