TabSTAR：一种面向含文本字段表格数据的表格基础模型
TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields
AI总结：
针对现有表格学习方法多采用静态目标无关文本表示、效果受限的问题，提出含语义目标感知表示的表格基础模型TabSTAR，通过解冻预训练文本编码器、输入目标标记实现迁移学习，在多基准分类任务上取得SOTA，预训练具数据集数量缩放定律。
摘要：
尽管深度学习已在诸多领域取得显著成功，但它在表格学习任务上的表现历来不佳，这类任务目前仍以梯度提升决策树为主导。不过，近期的进展正为表格基础模型的发展铺路，这类模型能够利用现实世界知识，在多样数据集间实现泛化，尤其是当数据包含自由文本时。虽然已有研究探索将语言模型能力融入表格任务，但现有大多数方法采用静态的、与目标无关的文本表示，这限制了其有效性。我们提出TabSTAR：一种具备语义目标感知表示的表格基础模型。TabSTAR旨在实现含文本特征的表格数据上的迁移学习，其架构不含数据集特定参数。它解冻预训练文本编码器，并将目标标记作为输入，为模型提供学习任务特定嵌入所需的上下文。在含文本特征的分类任务的已知基准上，TabSTAR在中型和大型数据集上均取得了最优性能，且其预训练阶段在数据集数量上呈现缩放定律，为进一步提升性能提供了路径。
英文摘要：
While deep learning has achieved remarkable success across many domains, it has historically underperformed on tabular learning tasks, which remain dominated by gradient boosting decision trees. However, recent advancements are paving the way for Tabular Foundation Models, which can leverage real-world knowledge and generalize across diverse datasets, particularly when the data contains free-text. Although incorporating language model capabilities into tabular tasks has been explored, most existing methods utilize static, target-agnostic textual representations, limiting their effectiveness. We introduce TabSTAR: a Tabular Foundation Model with Semantically Target-Aware Representations. TabSTAR is designed to enable transfer learning on tabular data with textual features, with an architecture free of dataset-specific parameters. It unfreezes a pretrained text encoder and takes as input target tokens, which provide the model with the context needed to learn task-specific embeddings. TabSTAR achieves state-of-the-art performance for both medium- and large-sized datasets across known benchmarks of classification tasks with text features, and its pretraining phase exhibits scaling laws in the number of datasets, offering a pathway for further performance improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
