arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.16801stat.MLcs.LG
收藏

当重复测量损害表格基础模型时

When Repeated Measurements Hurt Tabular Foundation Models

National University of Singapore(新加坡国立大学);Duke-NUS Medical School(杜克-国大医学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)

Yixiang Liu, Ruizhe Deng, Bibhas Chakraborty, Qiong Zhang, Yan Shuo Tan

AI总结:

本文研究表格基础模型(TFMs)对相同输入重复测量的响应,发现额外测量会损害部分输入的预测,分析其统计与计算层面原因,提出平均测量、重置计数等方法平衡重复测量的利弊。

摘要:

表格基础模型(Tabular Foundation Models, TFMs)在独立同分布(IID)基准上实现了最先进的性能,但它们对相同输入的重复测量有何反应?针对六个TFMs的合成与半合成实验表明,额外测量可提升被测输入的预测性能,却会降低其他输入的预测性能。损害在低噪声时最频繁,多个模型中无关特征会放大该损害;真实重复测量数据集也呈现类似模式。统计层面,PFN预训练针对的IID规则将每个重复输入视为关于输入分布的新证据,当先验将该分布与响应函数关联时,即使在精确推理下,由此产生的协议不匹配也会超过额外测量的益处。计算层面,干预手段识别出TabPFN-v3的注意力缩放中存在行计数依赖,这是损害的一个促成因素;重置计数可在无噪声线性诊断中大幅逆转失真,并削弱无关特征的放大作用。最后,本文评估了在保留重复测量益处的同时限制其损害的方法:在较大重复次数下,测量值平均通常优于单独提供测量值,而计数重置可同时抑制损害与有用增益。代码、保存的结果及复现说明可在该https URL获取。

英文摘要:

Tabular foundation models (TFMs) achieve state-of-the-art performance on IID benchmarks, but how do they respond to repeated measurements at the same inputs? Across six TFMs, synthetic and semi-synthetic experiments show that additional measurements can improve predictions at measured inputs while degrading them elsewhere. Harm is most frequent at low noise, and irrelevant features amplify it in several models; real repeated-measurement datasets show similar patterns. Statistically, the IID rule targeted by PFN pretraining treats each repeated input as fresh evidence about the input distribution. When the prior links this distribution to the response function, the resulting protocol mismatch can outweigh the benefit of additional measurements, even under exact inference. Computationally, interventions identify row-count dependence in TabPFN-v3's attention scaling as a contributor to harm. Resetting the count largely reverses distortion in a noiseless linear diagnostic and attenuates amplification by irrelevant features. Finally, we evaluate ways to retain the benefits of repeated measurements while limiting their harm. Averaging measurements usually outperforms supplying them separately at larger repetition counts, whereas count reset can suppress both harm and useful gains. Code, saved results and reproduction instructions are available at https://github.com/drizmiz/PFN-Repeated-Measurement-Supp

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑