arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06924cs.LG
收藏

在已知群体之外实现鲁棒性:低秩适应

Rank-Constrained Adaptation for Reliable Real-World Performance

Department of Electrical Engineering \& Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA;Harvard School of Medicine, Harvard, Boston, MA, USA

Abinitha Gourabathina, Hyewon Jeong, Teya Bergamaschi, Marzyeh Ghassemi, Collin Stultz

AI总结:

LEIA通过低秩调整分类器logits,在无需预设群体标签的情况下提升群体鲁棒性,适用于未知亚群体的场景。

摘要:

深度学习模型训练以优化平均准确率时,常常在特定亚群体上出现系统性失败。在现实世界中,受此类差异影响最大的亚群体往往是未标记或未知的,这促使开发出在敏感亚群体上表现优异但无需预设的 方法。然而,现有群体鲁棒方法通常假设已知相关亚群体,利用群体注释进行训练或模型选择。我们提出了低秩误差感知适应(LEIA),一种简单的两阶段方法,通过在表示空间中识别一个低维子空间(模型误差集中)来提高群体鲁棒性。LEIA通过低秩调整分类器logits来限制适应到这个误差感知子空间,直接针对潜在的失败模式,而无需修改主干网络或要求群体标签。在五个真实世界数据集上,我们分析了三种设置下的群体鲁棒性:(1)完全不了解亚群体相关性,(2)部分了解亚群体相关性,(3)完全了解亚群体相关性。在所有设置中,LEIA一致提高了最差群体的表现,同时保持快速、参数高效且对超参数选择具有鲁棒性。

英文摘要:

Deep learning models trained to optimize average accuracy often exhibit systematic failures on particular subpopulations. In real-world settings like healthcare, the subpopulations most affected by such disparities are frequently unlabeled, partially observed, or not known in advance. Existing group-robust methods typically assume prior knowledge of the relevant subgroups, using group annotations for training, validation, or model selection. We propose Misclassification Aware Rank-Limited Adaptation (MARLA), a parameter-efficient method for improving worst group performance without explicit subgroup annotations. MARLA leverages an ERM-trained model by calculating the model's misclassification probability scores on a held-out adaptation set to identify a low-dimensional subspace where errors concentrate. We then learn a rank-restricted additive correction to the classifier logits within that subspace. Across seven real-world datasets, we evaluate group robustness under three settings: no knowledge of subgroup relevance, partial knowledge of subgroup relevance, and full knowledge of subgroup relevance. MARLA improves worst-group performance while remaining fast and parameter-efficient, with data-guided hyperparameter selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑