通过大语言模型评估解释：超越传统用户研究
Evaluating Explanations Through LLMs: Beyond Traditional User Studies
AI总结：
本文提出用七个大语言模型在多种设置下模拟人类参与者，复现比较反事实与因果解释的用户研究，证明LLM可作为可扩展、低成本的定性XAI评估替代方案。
摘要：
随着人工智能在医疗等领域变得至关重要，可解释人工智能（XAI）工具对于信任与透明度不可或缺。然而，用于评估这些工具的传统用户研究往往成本高昂、耗时且难以扩展。本文探讨使用大语言模型（LLM）模拟人类参与者，以简化XAI评估。我们复现了一项比较反事实解释与因果解释的用户研究，在多种设置下使用七个LLM模拟人类参与者。结果表明：（i）LLM能够复现原始研究的大多数结论；（ii）不同LLM在结果中表现出不同程度的一致性；（iii）LLM记忆和输出可变性等实验因素会影响其与人类回答的一致性。这些初步发现表明，LLM可提供一种可扩展且具有成本效益的方式，以简化定性XAI评估。
英文摘要：
As AI becomes fundamental in sectors like healthcare, explainable AI (XAI) tools are essential for trust and transparency. However, traditional user studies used to evaluate these tools are often costly, time consuming, and difficult to scale. In this paper, we explore the use of Large Language Models (LLMs) to replicate human participants to help streamline XAI evaluation. We reproduce a user study comparing counterfactual and causal explanations, replicating human participants with seven LLMs under various settings. Our results show that (i) LLMs can replicate most conclusions from the original study, (ii) different LLMs yield varying levels of alignment in the results, and (iii) experimental factors such as LLM memory and output variability affect alignment with human responses. These initial findings suggest that LLMs could provide a scalable and cost-effective way to simplify qualitative XAI evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
