事后可解释性方法中的可预测性与可理解性：一项以用户为中心的分析
Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis
AI总结：
本文通过用户研究评估 LIME 和 SHAP 的可理解性与可预测性，发现决策边界附近样本会降低 SHAP 可理解性，而反事实解释和错误分类能显著提升用户对模型决策的理解，并据此提出设计建议。
摘要：
事后可解释性方法旨在阐明黑盒机器学习模型的预测。然而，用户能在多大程度上理解所提供的解释，以及这些解释是否提高了用户预测模型行为的能力，目前仍很不清楚。我们通过开展一项用户研究来探讨这一问题，评估两种广泛使用的工具 LIME 和 SHAP 的可理解性与可预测性。此外，我们还研究了反事实解释和错误分类对用户理解和预测模型行为的能力的影响。我们发现，当对靠近模型决策边界的样本提供解释时，SHAP 的可理解性显著降低。此外，我们发现反事实解释和错误分类能够显著提高用户对机器学习模型如何做出决策的理解。基于我们的发现，我们还为未来具有更高可理解性和可预测性的事后可解释性方法推导出设计建议。
英文摘要：
Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model's decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
