破解可解释人工智能（XAI）中的代理模型方法
Hacking a surrogate model approach to XAI
AI总结：
本研究针对可解释AI中代理模型的近似可靠性问题，揭示黑盒ADM系统的歧视模式可被人为隐藏在代理决策树深层，且现实场景下更易隐藏，该发现可推广至其他代理模型。
摘要：
近年来，高度复杂人工智能系统的新应用数量大幅增长。算法决策系统（ADMs）便是这类应用之一，其中人工智能系统取代了人类专家的决策流程。作为保障此类系统公平性与透明度的一种途径，可解释人工智能（XAI）的重要性日益凸显。实现可解释性的一种变体是代理模型，即基于黑盒模型的输入-输出关系训练一个新的、更简单的机器学习模型的思路。例如，这个更简单的机器学习模型可以是决策树，人们通常认为决策树对人类而言直观易懂。然而，目前对于代理模型对黑盒的近似效果如何，人们还缺乏深入了解。\n我们的核心假设是，一种优秀的代理模型方法应当能够将这种歧视性行为暴露给人类；在本研究开展之前，我们曾假设代理决策树会在其最初的几层中识别出这种模式。但在本文中我们证明，即便受歧视的子群体——在其他所有类别上均无差异——从未从黑盒ADM系统获得过一次正面决策，系统操作员也可以将对应的群体归属问题下推到任意想要的深层层级。\n随后我们对这一发现进行了推广，精确定位了歧视性问题在决策树中被询问的确切层级，并证明在更贴近现实的场景中（即歧视仅发生在部分弱势群体成员身上时），隐藏这类歧视的可行性甚至更高。\n我们的方法可以轻松推广到其他类型的代理模型。
英文摘要：
In recent years, the number of new applications for highly complex AI systems has risen significantly. Algorithmic decision-making systems (ADMs) are one of such applications, where an AI system replaces the decision-making process of a human expert. As one approach to ensure fairness and transparency of such systems, explainable AI (XAI) has become more important. One variant to achieve explainability are surrogate models, i.e., the idea to train a new simpler machine learning model based on the input-output-relationship of a black box model. The simpler machine learning model could, for example, be a decision tree, which is thought to be intuitively understandable by humans. However, there is not much insight into how well the surrogate model approximates the black box. Our main assumption is that a good surrogate model approach should be able to bring such a discriminating behavior to the attention of humans; prior to our research we assumed that a surrogate decision tree would identify such a pattern on one of its first levels. However, in this article we show that even if the discriminated subgroup - while otherwise being the same in all categories - does not get a single positive decision from the black box ADM system, the corresponding question of group membership can be pushed down onto a level as low as wanted by the operator of the system. We then generalize this finding to pinpoint the exact level of the tree on which the discriminating question is asked and show that in a more realistic scenario, where discrimination only occurs to some fraction of the disadvantaged group, it is even more feasible to hide such discrimination. Our approach can be generalized easily to other surrogate models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
