arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.04712cs.LG
收藏

通过聚类影响嵌入进行错误发现

Error Discovery by Clustering Influence Embeddings

Meta;Prescient Design / Genentech(Prescient Design / 基因泰克);Cornell University(康奈尔大学)

Fulton Wang, Julius Adebayo, Sarah Tan, Diego Garcia-Olano, Narine Kokhlikyan

AI总结:

本文提出切片发现方法 InfEmbed,利用影响函数构建影响嵌入并进行 K-Means 聚类,以发现因相同原因出错的连贯切片;实验表明其在2个基准上优于现有方法,并适用于模型调试。

摘要:

我们提出了一种用于识别模型表现不佳的测试示例组(即切片)的方法,这项任务如今被称为切片发现。我们将连贯性——即一个切片内的错误预测应当因相同原因而出错这一要求——形式化为任何切片发现方法都应满足的关键属性。随后,我们利用影响函数推导出一种新的切片发现方法 InfEmbed,该方法通过返回其示例受到训练数据相似影响的切片来满足连贯性。InfEmbed 非常简单,包括对我们称之为影响嵌入的一种新颖表示应用 K-Means 聚类。我们表明,InfEmbed 在 2 个基准测试上优于当前最先进的方法,并且在多个案例研究中对模型调试有效。

英文摘要:

We present a method for identifying groups of test examples -- slices -- on which a model under-performs, a task now known as slice discovery. We formalize coherence -- a requirement that erroneous predictions, within a slice, should be wrong for the same reason -- as a key property that any slice discovery method should satisfy. We then use influence functions to derive a new slice discovery method, InfEmbed, which satisfies coherence by returning slices whose examples are influenced similarly by the training data. InfEmbed is simple, and consists of applying K-Means clustering to a novel representation we deem influence embeddings. We show InfEmbed outperforms current state-of-the-art methods on 2 benchmarks, and is effective for model debugging across several case studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑