AI 中文总结
本文针对表格基础模型（TFMs），提出其预测是否可由任意联合分布生成的问题，设定边际一致性与分解一致性要求，发现所有评估的TFMs均违反这两项要求。
AI 中文摘要
表格基础模型（Tabular Foundation Models, TFMs）是当前表格预测问题的最优方法，被构建为基于预训练先验近似贝叶斯后验预测分布的Transformer模型，这类单变量预测器可通过自回归方式采样一个目标变量并将其加入特征，转换为多变量预测器。然而，所得联合分布的忠实性尚未得到研究；此外，至少在真实数据集上无法直接评估TFMs相对于后验本身的性能，因为真实分布未知。因此，本文提出一个不同的问题：模型的预测结果是否可由任意联合分布生成？为回答该问题，设定了模型必须满足的两个要求：一是边际一致性，要求边缘化条件概率等于直接预测的边际概率；二是分解一致性，要求不同分解顺序产生的联合分布相等。本文在所有数据集上对分类和回归任务评估的所有TFMs均违反了这两个要求。
英文摘要
Tabular Foundation Models (TFMs) are currently the best approach to tabular prediction problems. They are constructed as transformers that approximate the Bayesian posterior predictive distribution based on a pre-training prior. These univariate predictors can be converted into multivariate ones autoregressively by sampling one target and adding it to the features. However, the faithfulness of the resulting joint has not been investigated. Furthermore, TFMs cannot be evaluated against the posterior itself, at least not on real-world datasets, because the ground-truth distribution is unknown. We therefore propose asking a different question: could a model's predictions result from any joint distribution? To answer this question, we pose two requirements that any such model must satisfy. The first is marginalization consistency, which demands that marginalized conditionals are equal to directly predicted marginals. The second is factorization consistency, which demands that different factorization orders result in equal joint distributions. Every TFM that we evaluate violates both of these requirements for both classification and regression across all datasets.
