保镖问题：对远程解释能力的挑战
The Bouncer Problem: Challenges to Remote Explainability
摘要：
可解释性的概念是为了满足社会对机器学习决策透明度的要求。这个概念很简单：像人类一样，算法应该解释其决策背后的基本原理，以便评估其公平性。虽然这种方法在本地环境中很有前途（例如，在训练时的调试过程中解释模型），但我们认为这种推理不能简单地在远程环境中转换，其中服务提供商的训练模型只能通过其API访问。这是有问题的，因为它恰恰构成了从社会角度来看需要透明度的目标用例。通过与俱乐部保镖（这可能会提供不真实的解释后，客户拒绝）的类比，我们表明，提供解释不能阻止远程服务说谎的真正原因，导致其决定。更确切地说，我们证明了远程解释单个解释的不可能性，通过构建一个攻击的解释，隐藏歧视性的功能，查询用户。我们提供了这种攻击的示例实现。然后，我们表明，一个观察员发现的攻击，试图找到不连贯的几种解释的概率是低的实际设置。这破坏了一般意义上的远程可解释性的概念。
英文摘要：
The concept of explainability is envisioned to satisfy society's demands for transparency on machine learning decisions. The concept is simple: like humans, algorithms should explain the rationale behind their decisions so that their fairness can be assessed. While this approach is promising in a local context (e.g. to explain a model during debugging at training time), we argue that this reasoning cannot simply be transposed in a remote context, where a trained model by a service provider is only accessible through its API. This is problematic as it constitutes precisely the target use-case requiring transparency from a societal perspective. Through an analogy with a club bouncer (which may provide untruthful explanations upon customer reject), we show that providing explanations cannot prevent a remote service from lying about the true reasons leading to its decisions. More precisely, we prove the impossibility of remote explainability for single explanations, by constructing an attack on explanations that hides discriminatory features to the querying user. We provide an example implementation of this attack. We then show that the probability that an observer spots the attack, using several explanations for attempting to find incoherences, is low in practical settings. This undermines the very concept of remote explainability in general.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
