arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.17583cs.AIcs.LGcs.LOmath.CT
收藏

迈向XAI的组合可解释性

Towards Compositional Interpretability for XAI

Quantinuum

Sean Tull, Robin Lorenz, Stephen Clark, Ilyas Khan, Bob Coecke

AI总结:

提出基于范畴论的组合模型框架,通过形式化字符串图统一分析多种AI模型的组合结构与可解释性,定义组合可解释(CI)模型并展示其在计算推断与图示解释方面的优势。

摘要:

人工智能(AI)目前主要基于缺乏可解释性的黑盒机器学习模型。可解释人工智能(XAI)领域致力于解决这一主要问题,这在金融、法律和医疗等高风险领域至关重要。我们提出了一种基于范畴论定义AI模型及其可解释性的方法。为此,我们采用组合模型的概念,该概念通过形式化字符串图来审视模型,这些图捕获了模型的抽象结构及其具体实现。这种全面的视角涵盖了确定性、概率性和量子模型。我们将广泛的AI模型作为组合模型进行比较,包括线性模型和基于规则的模型、(循环)神经网络、transformers、VAEs,以及因果模型和DisCoCirc模型。接下来,我们根据模型的组合结构给出了模型解释的定义,演示了如何分析模型的可解释性,并利用这一点澄清了XAI中的常见主题。我们发现,标准“内在可解释”模型之所以如此透明,通过图示最为清晰地展现出来。这引导我们得出更一般的组合可解释(CI)模型概念,这些模型还包括因果模型、概念空间模型和DisCoCirc模型。接下来,我们展示了CI模型的可解释性优势。首先,它们的组合结构可能允许计算其他感兴趣的量,并可能通过匹配其结构来促进从模型到被建模现象的推断。其次,它们允许基于影响约束、图示手术和重写解释来对其行为进行图示解释。最后,我们讨论了该方法的许多未来方向,提出了在实践中如何学习这种有意义的结构化模型的问题。

英文摘要:

Artificial intelligence (AI) is currently based largely on black-box machine learning models which lack interpretability. The field of eXplainable AI (XAI) strives to address this major concern, being critical in high-stakes areas such as the finance, legal and health sectors. We present an approach to defining AI models and their interpretability based on category theory. For this we employ the notion of a compositional model, which sees a model in terms of formal string diagrams which capture its abstract structure together with its concrete implementation. This comprehensive view incorporates deterministic, probabilistic and quantum models. We compare a wide range of AI models as compositional models, including linear and rule-based models, (recurrent) neural networks, transformers, VAEs, and causal and DisCoCirc models. Next we give a definition of interpretation of a model in terms of its compositional structure, demonstrating how to analyse the interpretability of a model, and using this to clarify common themes in XAI. We find that what makes the standard 'intrinsically interpretable' models so transparent is brought out most clearly diagrammatically. This leads us to the more general notion of compositionally-interpretable (CI) models, which additionally include, for instance, causal, conceptual space, and DisCoCirc models. We next demonstrate the explainability benefits of CI models. Firstly, their compositional structure may allow the computation of other quantities of interest, and may facilitate inference from the model to the modelled phenomenon by matching its structure. Secondly, they allow for diagrammatic explanations for their behaviour, based on influence constraints, diagram surgery and rewrite explanations. Finally, we discuss many future directions for the approach, raising the question of how to learn such meaningfully structured models in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑