以人为中心、与人类共同开展的AI可解释性评估：一项全面的实证研究
User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study
AI总结：
本研究通过39名不同背景参与者的实证访谈，评估XGBClassifier生成的XAI解释的可理解性，揭示现有XAI方法局限，并提出面向不同AI利益相关者需求的新设计原则与跨学科评估方法。
摘要：
本研究属于人本人工智能（HCAI）领域，重点报告了对常用可解释人工智能（XAI）算法进行以用户为中心的评估结果，具体考察人类如何理解这些算法提供的解释并与之交互。为此，我们采用了一种多学科方法，引入社会科学领域最先进的研究方法，以测量由先进机器学习模型——具体为梯度提升分类器（XGBClassifier）——所生成解释的可理解性。我们开展了一项大规模实证用户研究，对来自三个不同群体的39名参与者进行了访谈，这些群体在数据科学、数据可视化以及与训练机器学习模型所用数据集相关的领域知识方面具有不同的专业水平。参与者被问及一系列问题，以评估他们对模型解释的理解程度。为确保可复现性，我们使用来自加州大学欧文分校机器学习库的公开数据集构建了模型，该数据集关注可食用与不可食用蘑菇。我们的研究结果揭示了现有XAI方法的局限性，并证实需要新的设计原则和评估技术，以应对不同类别AI利益相关者的特定信息需求和用户视角。我们相信，本研究的成果以及我们开发的跨学科方法能够成功适配多种数据类型和用户画像，从而促进对话并回应HCAI研究中的机遇。为支持这一点，我们将公开本研究产生的数据。
英文摘要：
This study is located in the Human-Centered Artificial Intelligence (HCAI) and focuses on the results of a user-centered assessment of commonly used eXplainable Artificial Intelligence (XAI) algorithms, specifically investigating how humans understand and interact with the explanations provided by these algorithms. To achieve this, we employed a multi-disciplinary approach that included state-of-the-art research methods from social sciences to measure the comprehensibility of explanations generated by a state-of-the-art lachine learning model, specifically the Gradient Boosting Classifier (XGBClassifier). We conducted an extensive empirical user study involving interviews with 39 participants from three different groups, each with varying expertise in data science, data visualization, and domain-specific knowledge related to the dataset used for training the machine learning model. Participants were asked a series of questions to assess their understanding of the model's explanations. To ensure replicability, we built the model using a publicly available dataset from the UC Irvine Machine Learning Repository, focusing on edible and non-edible mushrooms. Our findings reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders. We believe that the results of our research and the cross-disciplinary methodology we developed can be successfully adapted to various data types and user profiles, thus promoting dialogue and address opportunities in HCAI research. To support this, we are making the data resulting from our study publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
