arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.11440cs.HCcs.CYcs.LG
收藏

模型可解释性中的人为因素:行业实践、挑战和需求

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

New York University(纽约大学);Northwestern University(西北大学)

Sungsoo Ray Hong, Jessica Hullman, Enrico Bertini

摘要:

随着机器学习(ML)模型在产品开发和数据驱动决策过程中的应用在许多领域变得越来越普遍,人们对构建性能良好的模型的关注已经越来越多地转移到了解他们的模型是如何工作的。虽然在HCI,ML等研究社区中,对模型可解释性的学术兴趣迅速增长,但对于从业者如何感知并旨在在现有工作流程的背景下提供可解释性,人们知之甚少。这种对可解释性缺乏理解的做法可能会阻止可解释性研究解决重要的需求,或导致不切实际的解决方案。为了弥合这一差距,我们对行业从业者进行了22次半结构化访谈,以了解他们在规划、构建和使用模型时如何构思和设计可解释性。基于对结果的定性分析,我们区分了大量使用ML模型的组织中存在的可解释性角色、流程、目标和策略。从我们的分析中出现的可解释性工作的特征表明,模型的可解释性经常涉及不同角色的人之间的合作和心理模型比较,通常旨在建立信任,不仅在人与模型之间,而且在组织内的人之间。我们提出了设计的影响,讨论了从业者在实践中面临的可解释性挑战和文献中提出的方法之间的差距,突出了可以更好地满足现实世界需求的可能的研究方向。

英文摘要:

As the use of machine learning (ML) models in product development and data-driven decision-making processes became pervasive in many domains, people's focus on building a well-performing model has increasingly shifted to understanding how their model works. While scholarly interest in model interpretability has grown rapidly in research communities like HCI, ML, and beyond, little is known about how practitioners perceive and aim to provide interpretability in the context of their existing workflows. This lack of understanding of interpretability as practiced may prevent interpretability research from addressing important needs, or lead to unrealistic solutions. To bridge this gap, we conducted 22 semi-structured interviews with industry practitioners to understand how they conceive of and design for interpretability while they plan, build, and use their models. Based on a qualitative analysis of our results, we differentiate interpretability roles, processes, goals and strategies as they exist within organizations making heavy use of ML models. The characterization of interpretability work that emerges from our analysis suggests that model interpretability frequently involves cooperation and mental model comparison between people in different roles, often aimed at building trust not only between people and models but also between people within the organization. We present implications for design that discuss gaps between the interpretability challenges that practitioners face in their practice and approaches proposed in the literature, highlighting possible research directions that can better address real-world needs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑