Learning-From-Disagreement：一个模型比较和可视化分析框架
Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework
摘要：
随着每天产生的分类模型数量的快速增长，也引入了许多模型解释和比较解决方案。例如，LIME和SHAP可以解释哪些输入特征对分类器的输出预测贡献更大。不同的数值度量（例如，准确度）可以用于容易地比较两个分类器。然而，很少有作品可以解释的贡献，一个数据特征的分类器相比，它的贡献，另一个分类器。这种比较解释有助于揭示两种分类器之间的根本差异，选择不同特征条件下的分类器，更好地集成两种分类器。为了实现这一点，我们提出了一个从分歧中学习（LFD）框架来直观地比较两个分类模型。具体来说，LFD识别来自两个比较分类器的预测不一致的数据实例，并训练一个学习器从不一致的实例中学习。由于这两个分类器的训练特征可能不可用，我们通过基于分类器的某些假设提出的一组元特征来训练分类器，以探测它们的行为。通过使用不同元特征的SHAP值解释训练的分类器，我们为比较的分类器提供了可操作的见解。此外，我们引入了多个指标，从不同的角度来分析元功能的重要性。有了这些指标，人们可以很容易地识别出两个分类器中具有最互补行为的元特征，并使用它们来更好地集成分类器。我们专注于金融服务和广告行业的二进制分类模型，以证明我们提出的框架和可视化的有效性。
英文摘要：
With the fast-growing number of classification models being produced every day, numerous model interpretation and comparison solutions have also been introduced. For example, LIME and SHAP can interpret what input features contribute more to a classifier's output predictions. Different numerical metrics (e.g., accuracy) can be used to easily compare two classifiers. However, few works can interpret the contribution of a data feature to a classifier in comparison with its contribution to another classifier. This comparative interpretation can help to disclose the fundamental difference between two classifiers, select classifiers in different feature conditions, and better ensemble two classifiers. To accomplish it, we propose a learning-from-disagreement (LFD) framework to visually compare two classification models. Specifically, LFD identifies data instances with disagreed predictions from two compared classifiers and trains a discriminator to learn from the disagreed instances. As the two classifiers' training features may not be available, we train the discriminator through a set of meta-features proposed based on certain hypotheses of the classifiers to probe their behaviors. Interpreting the trained discriminator with the SHAP values of different meta-features, we provide actionable insights into the compared classifiers. Also, we introduce multiple metrics to profile the importance of meta-features from different perspectives. With these metrics, one can easily identify meta-features with the most complementary behaviors in two classifiers, and use them to better ensemble the classifiers. We focus on binary classification models in the financial services and advertising industry to demonstrate the efficacy of our proposed framework and visualizations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
