arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01259cs.LGcs.AI
收藏

Concept Bottleneck Models 是否尊重 Localities?

Do Concept Bottleneck Models Respect Localities?

Carnegie Mellon University(卡内基梅隆大学);University of Cambridge(剑桥大学)

Naveen Raman, Mateo Espinosa Zarlenga, Juyeon Heo, Mateja Jamnik

AI总结:

本文研究了 Concept-based models 是否利用相关特征进行预测(即 locality),构建了三个指标并结合理论分析发现许多现有模型未能尊重 localities,进而提出缓解建议。

摘要:

Concept-based explainability methods 使用人类可理解的中间量来为机器学习模型生成解释。这些方法假设概念预测能帮助理解模型的内部推理。在本文中,我们通过分析概念预测器是否利用“相关”特征进行预测来评估该假设的真实程度,我们将此称为 locality。未能尊重 localities 的 Concept-based models 也无法被解释,因为概念预测基于虚假特征,使得对概念预测的解释变得空洞。为了评估 Concept-based models 是否尊重 localities,我们构建并使用了三个指标来表征模型何时尊重 localities,并以理论结果补充我们的分析。我们的每个指标捕获不同的扰动概念,并评估扰动“无关”特征是否会影响概念预测器做出的预测。我们发现实践中使用的许多 Concept-based models 未能尊重 localities,因为概念预测器无法始终清晰地区分不同概念。基于这些发现,我们提出了一些建议以缓解此问题。

英文摘要:

Concept-based explainability methods use human-understandable intermediaries to produce explanations for machine learning models. These methods assume concept predictions can help understand a model's internal reasoning. In this work, we assess the degree to which such an assumption is true by analyzing whether concept predictors leverage "relevant" features to make predictions, a term we call locality. Concept-based models that fail to respect localities also fail to be explainable because concept predictions are based on spurious features, making the interpretation of the concept predictions vacuous. To assess whether concept-based models respect localities, we construct and use three metrics to characterize when models respect localities, complementing our analysis with theoretical results. Each of our metrics captures a different notion of perturbation and assess whether perturbing "irrelevant" features impacts the predictions made by a concept predictors. We find that many concept-based models used in practice fail to respect localities because concept predictors cannot always clearly distinguish distinct concepts. Based on these findings, we propose suggestions for alleviating this issue.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑