arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.06759cs.LG
收藏

基于梯度空间聚类的离群鲁棒群推理

Outlier-Robust Group Inference via Gradient Space Clustering

University of Wisconsin-Madison(威斯康星大学麦迪逊分校);IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);Massachusetts Institute of Technology(麻省理工学院)

Yuchen Zeng, Kristjan Greenewald, Kangwook Lee, Justin Solomon, Mikhail Yurochkin

摘要:

传统的机器学习模型专注于在整体训练分布上实现良好的性能,但它们在少数群体上往往表现不佳。现有的方法可以提高最差组的性能,但它们可能有几个限制:(i)它们需要组注释,这通常是昂贵的,有时不可行的获得,和/或(ii)它们是敏感的离群值。大多数相关的作品未能同时解决这两个问题,因为他们专注于少数群体和离群者的相互冲突的观点。我们解决的问题,学习组注释中存在的离群值的模型参数的梯度空间中的数据进行聚类。我们表明,梯度空间中的数据具有更简单的结构,同时保留了少数群体和离群点的信息,使其适用于标准的聚类方法,如DBSCAN。大量的实验表明,我们的方法显着优于国家的最先进的组识别和下游最坏的组性能。

英文摘要:

Traditional machine learning models focus on achieving good performance on the overall training distribution, but they often underperform on minority groups. Existing methods can improve the worst-group performance, but they can have several limitations: (i) they require group annotations, which are often expensive and sometimes infeasible to obtain, and/or (ii) they are sensitive to outliers. Most related works fail to solve these two issues simultaneously as they focus on conflicting perspectives of minority groups and outliers. We address the problem of learning group annotations in the presence of outliers by clustering the data in the space of gradients of the model parameters. We show that data in the gradient space has a simpler structure while preserving information about minority groups and outliers, making it suitable for standard clustering methods like DBSCAN. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art both in terms of group identification and downstream worst-group performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑