操纵和测量模型的可解释性
Manipulating and Measuring Model Interpretability
摘要：
随着机器学习模型越来越多地用于辅助决策，甚至在高风险领域，人们对开发可解释的模型越来越感兴趣。虽然已经提出了许多假定可解释的模型，但相对较少的实验研究调查这些模型是否达到了预期的效果，例如让人们更密切地关注模型的预测，当这对他们有利时，或者让他们能够检测到模型是否犯了错误。我们提出了一系列预先注册的实验（N= 3，800），在这些实验中，我们向参与者展示了功能相同的模型，这些模型只在两个因素上有所不同，这两个因素通常被认为使机器学习模型或多或少地可解释：特征的数量和模型的透明度（即，模型内部是透明的还是黑盒）。可以预见的是，那些看到一个清晰的模型，几乎没有特征的参与者可以更好地模拟模型的预测。然而，我们没有发现参与者更密切地遵循其预测。此外，向参与者展示一个清晰的模型意味着他们不太能够检测和纠正模型的相当大的错误，这似乎是由于信息过载。这些违反直觉的发现强调了在开发可解释的模型时测试对直觉的重要性。
英文摘要：
With machine learning models being increasingly used to aid decision making even in high-stakes domains, there has been a growing interest in developing interpretable models. Although many supposedly interpretable models have been proposed, there have been relatively few experimental studies investigating whether these models achieve their intended effects, such as making people more closely follow a model's predictions when it is beneficial for them to do so or enabling them to detect when a model has made a mistake. We present a sequence of pre-registered experiments (N=3,800) in which we showed participants functionally identical models that varied only in two factors commonly thought to make machine learning models more or less interpretable: the number of features and the transparency of the model (i.e., whether the model internals are clear or black box). Predictably, participants who saw a clear model with few features could better simulate the model's predictions. However, we did not find that participants more closely followed its predictions. Furthermore, showing participants a clear model meant that they were less able to detect and correct for the model's sizable mistakes, seemingly due to information overload. These counterintuitive findings emphasize the importance of testing over intuition when developing interpretable models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
