超越训练模型：编译GNN以构建可靠的解释器基准
Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark
浏览论文内容
中文总结 AI 辅助
本研究提出Gracr编译器，将模态逻辑公式编译为GNN权重，构建GracrBench基准，以精确真实标签评估解释器，发现多数解释器对间接影响不稳健。
中文摘要 AI 辅助
图神经网络（GNN）的解释器通常通过其合理性进行评估，即其解释在多大程度上恢复了预定义的真实标签，例如数据中植入的基序。该协议隐含地假设在此类数据上训练的GNN依赖于预期的基序。尽管先前的工作已质疑此假设，但合理性评估仍然广泛使用。首先，我们表明该假设在几个广泛使用的基准上被违反，例如，仅凭度数统计就足以解决任务。然后，我们通过用编译替代训练来消除这一混淆因素。我们通过引入$\mathsf{Gracr}$来实现这一点，这是第一个将分级模态逻辑公式翻译成GNN权重的编译器，从而产生能够复现相应公式行为的模型。由于模型的行为现在通过构造已知，我们可以正式定义其真实标签解释并精确计算。在此基础上，我们引入了$\mathsf{Gracr}\mathsf{Bench}$，一个编译GNN的基准，用于针对这一精确真实标签评估解释器。在六个任务上的十一个解释器的实验显示了其用于细粒度诊断评估的有效性：值得注意的是，我们发现大多数解释器对间接影响或同一公式的替代实现不稳健。这些结果将$\mathsf{Gracr}\mathsf{Bench}$定位为图事后可解释性的新颖、严格的评估环境。
英文摘要
Explainers for Graph Neural Networks (GNNs) are commonly evaluated by their plausibility, i.e., how well their explanations recover a predefined ground truth, such as a motif planted in the data. This protocol implicitly assumes that a GNN trained on such data relies on the intended motif. Although prior work has questioned this assumption, plausibility remains widespread. First, we show that the assumption is violated on several widely used benchmarks, where, e.g., degree statistics alone suffice to solve the task. Then, we remove this confounder by replacing training with compilation. We achieve this by introducing $\mathsf{Gracr}$, the first compiler translating graded modal logic formulas into GNN weights, yielding models that replicate the behaviour of the corresponding formulas. Since the behaviour of the model is now known by construction, we can define its ground truth explanation formally and compute it exactly. Building on this, we introduce $\mathsf{Gracr}\mathsf{Bench}$, a benchmark of compiled GNNs for the evaluation of explainers against this exact ground truth. Experiments on eleven explainers across six tasks show its effectiveness for fine-grained diagnostic evaluation: notably, we discover that most explainers are not robust to indirect influences or alternative implementations of the same formula. These results position $\mathsf{Gracr}\mathsf{Bench}$ as a novel, rigorous evaluation setting for graph post-hoc explainability.
发表机构
- University of Trento（特伦托大学）
- Sapienza University（罗马大学）
- Intesa Sanpaolo AI Research（意大利联合圣保罗银行人工智能研究院）
- Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
机构由 AI 辅助整理，请以论文原文为准。
