Feature attribution methods are popular in interpretable machine learning. These methods compute the attribution of each input feature to represent its importance, but there is no consensus on the definition of "attribution", leading to many competing methods with little systematic evaluation, complicated in particular by the lack of ground truth attribution. To address this, we propose a dataset modification procedure to induce such ground truth. Using this procedure, we evaluate three common methods: saliency maps, rationales, and attentions. We identify several deficiencies and add new perspectives to the growing body of evidence questioning the correctness and reliability of these methods applied on datasets in the wild. We further discuss possible avenues for remedy and recommend new attribution methods to be tested against ground truth before deployment. The code is available at https://github.com/YilunZhou/feature-attribution-evaluation
Generalizable Mixed-Precision Quantization via Attribution Rank Preservation
Tsinghua University（清华大学）；State Key Lab of Intelligent Technologies and Systems（智能技术与系统国家重点实验室）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
University at Buffalo, State University of New York（纽约州立大学布法罗分校）；West Virginia University（西弗吉尼亚大学）
AI创建的换脸视频，俗称Deepfakes，作为强大的模仿攻击，引起了广泛关注。现有的关于Deepfakes的研究主要集中在二进制检测，以区分真实和虚假视频。然而，确定假视频的特定生成模型也很重要，这可以帮助将其归因于取证调查的来源。在本文中，我们通过研究Deepfake视频的模型属性问题来填补这一空白。我们首先介绍了一个新的数据集，其中包含基于几个自动编码器模型的DeepFakes from Different Models（DFDM）。具体来说，在编码器、解码器、中间层、输入分辨率和压缩比方面存在差异的五种生成模型已被用于基于相同输入生成总共6，450个Deepfake视频。然后，我们将Deepfakes模型属性作为多类分类任务，并提出了一种基于空间和时间注意力的方法来探索新数据集中Deepfakes之间的差异。实验评估表明，大多数现有的Deepfakes检测方法在Deepfakes模型属性上失败，而所提出的方法在高质量的DFDM数据集上达到了70%以上的准确率。
Less is More: Fewer Interpretable Region via Submodular Subset Selection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
A Simple and Efficient Baseline for Data Attribution on Images
University of Maryland（马里兰大学）；New York University（纽约大学）；ELLIS Institute Tübingen（蒂宾根ELLIS研究所）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（蒂宾根人工智能中心）
Why Do Class-Dependent Evaluation Effects Occur with Time Series Feature Attributions? A Synthetic Data Investigation
Information Systems, Eindhoven University of Technology, Eindhoven, The Netherlands（信息系统，埃因霍温理工大学，埃因霍温，荷兰）；Eindhoven Artificial Intelligence Systems Institute, Eindhoven University of Technology, Eindhoven, The Netherlands（埃因霍温人工智能系统研究所，埃因霍温理工大学，埃因霍温，荷兰）；Human-Technology Interaction, Eindhoven University of Technology, Eindhoven, The Netherlands（人机交互，埃因霍温理工大学，埃因霍温，荷兰）
