发表机构
Columbia University; Cornell Tech（哥伦比亚大学; 康奈尔科技校区）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
MatrixFormer是一种预训练的矩阵原生Transformer，通过单次前向传播预测缺失项的完整分布，在零样本下于因果推断、表格填补和推荐系统等任务中达到竞争性能，成为通用矩阵补全基础模型。
AI 中文摘要
矩阵补全是表格数据填补到因果推断等问题的核心，然而现有的表格基础模型将其视为逐项预测，为每个目标重复上下文并丢弃矩阵的二维结构。我们提出MatrixFormer，一个预训练的矩阵原生Transformer，它在单次前向传播中为每个缺失项预测完整分布。MatrixFormer完全在合成低秩和潜在因子矩阵上训练，并涵盖多种缺失模式。在零样本且使用相同模型权重的情况下，MatrixFormer在因果推断面板数据任务、语言模型基准分数补全、表格数据填补和推荐系统矩阵补全上均取得了具有竞争力的性能。这些结果将MatrixFormer定位为用于矩阵补全的通用基础模型。
英文摘要
Matrix completion underlies problems from tabular imputation to causal inference, yet existing tabular foundation models treat it as entry-by-entry prediction, repeating context for every target and discarding the matrix's two-dimensional structure. We introduce MatrixFormer, a pre-trained matrix-native transformer that predicts a full distribution for every missing entry in a single forward pass. MatrixFormer is trained entirely on synthetic low-rank and latent-factor matrices under diverse missingness patterns. Applied zero-shot and with the same model weights, MatrixFormer achieves competitive performance on causal inference panel-data tasks, language-model benchmark-score completion, tabular imputation, and recommendation systems matrix completion. These results position MatrixFormer as a general-purpose foundation model for matrix completion.
Comments17 pages, 5 figures
