arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.08181cs.LG
收藏

通过棱镜审视TabPFN:对TabPFN及其内部表示的可解释性研究

TabPFN Through The Looking Glass: An interpretability study of TabPFN and its internal representations

Birla Institute of Technology(巴里尔理工学院)

Aviral Gupta, Armaan Sethi, Dhruv Kumar

AI总结:

本研究通过分析TabPFN模型的内部表示,揭示其如何编码结构化信息,提升模型决策的透明度和可信度。

摘要:

表格基础模型是为广泛表格数据任务预训练的模型。它们在各个领域都表现出强大的性能,但其内部表示和学习的概念仍然理解不足。这种可解释性不足使得研究这些模型如何处理和转换输入特征变得重要。在本工作中,我们分析了模型隐藏表示中编码的信息,并检查这些表示在各层之间的演变。我们进行了一系列探测实验,测试早期层中是否存在线性回归系数、复杂表达式的中间值以及最终答案。这些实验使我们能够推断模型内部执行的计算。我们的结果提供了证据,表明有意义且结构化的信息存储在表格基础模型的表示中。我们观察到清晰的信号,对应于模型预测过程中涉及的中间和最终量。这为模型如何细化其输入以及最终输出如何产生提供了见解。我们的发现有助于更深入地理解表格基础模型的内部机制。它们表明这些模型编码了具体且可解释的信息,这使我们更接近使它们的决策过程更加透明和可信。

英文摘要:

Tabular foundational models are pre-trained models designed for a wide range of tabular data tasks. They have shown strong performance across domains, yet their internal representations and learned concepts remain poorly understood. This lack of interpretability makes it important to study how these models process and transform input features. In this work, we analyze the information encoded inside the model's hidden representations and examine how these representations evolve across layers. We run a set of probing experiments that test for the presence of linear regression coefficients, intermediate values from complex expressions, and the final answer in early layers. These experiments allow us to reason about the computations the model performs internally. Our results provide evidence that meaningful and structured information is stored inside the representations of tabular foundational models. We observe clear signals that correspond to both intermediate and final quantities involved in the model's prediction process. This gives insight into how the model refines its inputs and how the final output emerges. Our findings contribute to a deeper understanding of the internal mechanics of tabular foundational models. They show that these models encode concrete and interpretable information, which moves us closer to making their decision processes more transparent and trustworthy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑