发表机构
imec-SMIT, Vrije Universiteit Brussel; KU Leuven; Augment, imec research group at KU Leuven; Computer Science Department, Pontificia Universidad Católica de Chile（imec-SMIT，布鲁塞尔自由大学; 荷语鲁汶大学; 荷语鲁汶大学imec研究组Augment; 智利天主教 Pontificia Universidad Católica de Chile 计算机科学系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种基于82项研究分析构建的XAI评估框架和36张卡片分类方法，帮助研究人员系统设计以人为中心的XAI评估，经五个项目测试，能简化流程并促进多学科综合评估。
AI 中文摘要
从以人为中心的方法评估可解释人工智能（XAI）系统，要求研究人员从众多评估维度和测量指标中进行选择，这通常是临时且零散的。本文介绍了一种方法，帮助人机交互（HCI）、计算机科学、设计和社会科学研究人员系统地评估XAI系统。该方法基于一个更新的XAI特定评估框架，该框架源自对82项研究的分析。利用这一框架，我们开发了一种包含36张卡片的卡片分类方法，帮助研究人员优先考虑相关的评估方面。该过程在两个研究小组（n = 13）的五个项目中进行了测试。XAI评估卡片可作为可打印附录获取，并附带一个在线存储库，收录了以往XAI研究的方法。尽管并非详尽无遗，我们的研究结果表明，卡片分类方法能够组织和简化评估过程的设计，鼓励在研究和开发中对XAI系统进行更全面和多学科的评估。
英文摘要
Evaluating explainable AI (XAI) systems from a human-centred approach requires researchers to select from numerous evaluation dimensions and measures, often in an ad hoc and fragmented manner. This paper introduces a method to help HCI, computer science, designers and social science researchers systematically evaluate XAI systems. The approach is based on an updated XAI-specific evaluation framework derived from an analysis of 82 studies. Using this framework, we developed a card-sorting method with 36 cards to help researchers prioritise relevant evaluation aspects. The process was tested with two research groups (n = 13) across five projects. The XAI Evaluation Cards are available as a printable appendix, along with an online repository of methods from previous XAI studies. Although not exhaustive, our findings indicate that the card-sorting approach can organise and streamline the design of the evaluation process, encouraging a more comprehensive and multidisciplinary assessment of XAI systems in research and development.
