发表机构
Pohang University of Science and Technology（浦项科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对子群体偏移下表格基础模型对弱势群体表现不佳的问题，提出无需真实群体标注的参数高效鲁棒适应框架DR-TFM，仅更新0.016%参数，在多个基准上显著提升最差群体准确率。
AI 中文摘要
尽管平均准确率很高，表格基础模型（TFMs）在子群体偏移下，即训练和部署时群体比例发生变化的情况下，可能对代表性不足的群体表现不佳。我们提出了DR-TFM，一种参数高效的分布鲁棒适应框架，该框架不需要真实的群体标注。DR-TFM通过微调现有的查询缩放网络或添加并训练一个查询缩放网络来调整对标注上下文示例的注意力，同时保持所有其他参数固定。我们使用从训练数据中得出的估计群体或源条件分布，以两个鲁棒目标实例化了该框架。对于TabPFN-3，适应仅更新预训练模型参数的0.016%。在五个表格基准上，DR-TFM实现的平均最差群体准确率显著高于预训练TFM和没有真实群体标注的比较鲁棒基线，同时保持了具有竞争力的平均群体准确率。DR-TFM还在ACS Income和另外四个TFM上提高了平均最差群体准确率。
英文摘要
Despite strong mean accuracy, tabular foundation models (TFMs) can perform poorly on underrepresented groups under subpopulation shift, where group proportions change between training and deployment. We propose DR-TFM, a parameter-efficient distributionally robust adaptation framework that requires no true group annotations. DR-TFM adjusts attention to labeled context examples by fine-tuning an existing query scaling network or adding and training one, while keeping all other parameters fixed. We instantiate the framework with two robust objectives using estimated groups or source conditional distributions derived from training data. For TabPFN-3, adaptation updates only 0.016% of the pretrained model's parameters. Across five tabular benchmarks, DR-TFM achieves substantially higher average worst-group accuracy than pretrained TFMs and the compared robust baselines without true group annotations, while maintaining competitive mean group accuracy. DR-TFM also improves average worst-group accuracy on ACS Income and across four additional TFMs.
Comments45 pages, 7 figures, including appendices
