Concept bottleneck models (CBMs) are a class of interpretable neural network models that predict the target response of a given input based on its high-level concepts. Unlike the standard end-to-end models, CBMs enable domain experts to intervene on the predicted concepts and rectify any mistakes at test time, so that more accurate task predictions can be made at the end. While such intervenability provides a powerful avenue of control, many aspects of the intervention procedure remain rather unexplored. In this work, we develop various ways of selecting intervening concepts to improve the intervention effectiveness and conduct an array of in-depth analyses as to how they evolve under different circumstances. Specifically, we find that an informed intervention strategy can reduce the task error more than ten times compared to the current baseline under the same amount of intervention counts in realistic settings, and yet, this can vary quite significantly when taking into account different intervention granularity. We verify our findings through comprehensive evaluations, not only on the standard real datasets, but also on synthetic datasets that we generate based on a set of different causal graphs. We further discover some major pitfalls of the current practices which, without a proper addressing, raise concerns on reliability and fairness of the intervention procedure.
TU Darmstadt(达姆施塔特工业大学);Hessian Center for Artificial Intelligence (hessian.AI)(黑森人工智能中心);Centre for Cognitive Science, TU Darmstadt(达姆施塔特工业大学认知科学中心);German Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)
Department of Applied Mathematics & Statistics(应用数学与统计学系);State University of New York at Stony Brook(石溪州立大学);Halıcıoğlu Data Science Institute(Halıcıoğlu数据科学研究所);University of California, San Diego(加州大学圣地亚哥分校);Thomas J. Watson Research Center(汤普森·J·沃森研究中心);IBM Research(IBM研究院);Department of Biomedical Informatics(生物医学信息学系)
Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off
University of Cambridge(剑桥大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);KU Leuven(鲁汶大学);University of Siena(锡耶纳大学);Babylon Health(巴比伦健康公司);Alan Turing Institute(艾伦·图灵研究所)
TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment
Department of Biomedical and Healthcare Data Engineering, Faculty of Engineering, Cairo University(生物医学与健康数据工程系,工程学院,开罗大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));University of Oldenburg(奥尔登堡大学)
SL-CBM: Enhancing Concept Bottleneck Models with Semantic Locality for Better Interpretability
Saarland University(萨尔大学);University of Chinese Academy of Sciences(中国科学院大学);Institute of Science Tokyo(东京科学大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Institute of Software, Chinese Academy of Science(中国科学院软件研究所)
Università della Svizzera Italiana(瑞士意大利大学);University of Liechtenstein(利赫泰森大学);Politecnico di Torino(托尼诺理工学院);IBM Research(IBM研究院);Scuola Universitaria Professionale della Svizzera Italiana(瑞士意大利专业大学)
Università della Svizzera Italiana(瑞士意大利语大学);University of Cambridge(剑桥大学);Scuola Normale Superiore(高等师范学院);Politecnico di Torino(都灵理工大学);University of Siena(锡耶纳大学);KU Leuven(鲁汶大学)
Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions
Korteweg-de Vries Institute for Mathematics, University of Amsterdam(阿姆斯特丹大学科特韦赫-德弗里斯数学研究所);Informatics Institute, University of Amsterdam(阿姆斯特丹大学信息学研究所)
Automatic Concept Extraction for Concept Bottleneck-based Video Classification
University of California Los Angeles(加州大学洛杉矶分校);University College London(伦敦大学学院);USC Information Sciences Institute(南加州大学信息科学研究所);Imperial College London(帝国理工学院);U.S. Army Research Laboratory(美国陆军研究实验室);U.S. Air Force Research Lab(美国空军研究实验室)
Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds
Center for Brain-Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China(脑启发计算研究中心,精密仪器系,清华大学,北京,中国);College of Computer Science and Technology, Zhejiang University, Hangzhou, China(计算机科学与技术学院,浙江大学,杭州,中国);Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China(彭途萨微纳米科学与技术研究院,厦门大学,厦门,中国)