arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.01888cs.HC
收藏

ConceptExplainer:从概念角度对深度神经网络的交互式解释

ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective

Jinbin Huang, Aditi Mishra, Bum Chul Kwon, Chris Bryan

摘要:

传统的适用于模型用户的深度学习可解释性方法无法在全局层面上解释网络行为,并且在提供细粒度解释方面不灵活。作为一种解决方案,基于概念的解释由于其人类的直观性和描述全局和局部模型行为的灵活性而受到关注。概念是一组类似有意义的像素,它们表达一个概念,嵌入在网络的潜在空间中,通常是手工生成的,但最近通过自动化方法发现了。不幸的是,所发现的概念的数量和多样性使得难以导航和理解概念空间。可视化分析可以通过支持概念空间的结构化导航和探索,为用户提供基于概念的模型行为见解,从而在弥合这些差距方面发挥重要作用。为此,我们设计,开发和验证ConceptExplainer,一个可视化的分析系统,使人们能够交互式地探测和探索概念空间,解释模型的行为在实例/类/全局级别。该系统通过迭代原型开发,以解决建模用户在解释深度学习模型的行为时面临的许多设计挑战。通过严格的用户研究,我们验证了ConceptExplainer如何支持这些挑战。同样,我们进行了一系列的使用场景,以演示系统如何支持跨各种任务和解释粒度的模型行为的交互式分析,例如识别对分类很重要的概念,识别训练数据中的偏差,以及理解如何在不同的和看似不同的类中共享概念。

英文摘要:

Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly meaningful pixels that express a notion, embedded within the network's latent space and have commonly been hand-generated, but have recently been discovered by automated approaches. Unfortunately, the magnitude and diversity of discovered concepts makes it difficult to navigate and make sense of the concept space. Visual analytics can serve a valuable role in bridging these gaps by enabling structured navigation and exploration of the concept space to provide concept-based insights of model behavior to users. To this end, we design, develop, and validate ConceptExplainer, a visual analytics system that enables people to interactively probe and explore the concept space to explain model behavior at the instance/class/global level. The system was developed via iterative prototyping to address a number of design challenges that model users face in interpreting the behavior of deep learning models. Via a rigorous user study, we validate how ConceptExplainer supports these challenges. Likewise, we conduct a series of usage scenarios to demonstrate how the system supports the interactive analysis of model behavior across a variety of tasks and explanation granularities, such as identifying concepts that are important to classification, identifying bias in training data, and understanding how concepts can be shared across diverse and seemingly dissimilar classes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑