Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models
Fraunhofer Heinrich-Hertz-Institute(弗劳恩霍夫海因里希·赫兹研究所);Technische Universität Berlin(柏林工业大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(BIFOLD——柏林学习与数据基础研究所)
Frederik Pahde, Maximilian Dreyer, Wojciech Samek, Sebastian Lapuschkin
最先进的机器学习模型经常学习嵌入在训练数据中的虚假相关性。这在部署这些模型进行高风险决策时带来了风险,例如在皮肤癌检测等医疗应用中。为了解决这个问题,我们提出了Reveal to Revise(R2R),这是一个包含整个可解释人工智能(XAI)生存周期的框架,使从业者能够迭代地识别,缓解和(重新)评估虚假的模型行为,只需最少的人工交互。在第一步(1)中,R2R通过发现属性中的离群值或通过检查模型学习的潜在概念来揭示模型的弱点。其次(2),在输入数据中检测并空间定位负责的伪影,然后利用该伪影来(3)修正模型行为。具体地说,我们应用RRR、CDEP和ClArC方法进行模型校正,(4)重新评估模型的性能和对伪影的剩余敏感性。使用两个用于黑色素瘤检测和骨龄估计的医学基准数据集,我们将我们的R2R框架应用于VGG,ResNet和EfficientNet架构,从而揭示和纠正真实的黑色素瘤内在伪影,以及受控环境中的合成变体。在完成XAI生存周期后,我们演示了多个R2R迭代以减轻不同的偏倚。代码可在https://github.com/maxdreyer/Reveal2Revise上获得。
英文摘要:
State-of-the-art machine learning models often learn spurious correlations embedded in the training data. This poses risks when deploying these models for high-stake decision-making, such as in medical applications like skin cancer detection. To tackle this problem, we propose Reveal to Revise (R2R), a framework entailing the entire eXplainable Artificial Intelligence (XAI) life cycle, enabling practitioners to iteratively identify, mitigate, and (re-)evaluate spurious model behavior with a minimal amount of human interaction. In the first step (1), R2R reveals model weaknesses by finding outliers in attributions or through inspection of latent concepts learned by the model. Secondly (2), the responsible artifacts are detected and spatially localized in the input data, which is then leveraged to (3) revise the model behavior. Concretely, we apply the methods of RRR, CDEP and ClArC for model correction, and (4) (re-)evaluate the model's performance and remaining sensitivity towards the artifact. Using two medical benchmark datasets for Melanoma detection and bone age estimation, we apply our R2R framework to VGG, ResNet and EfficientNet architectures and thereby reveal and correct real dataset-intrinsic artifacts, as well as synthetic variants in a controlled setting. Completing the XAI life cycle, we demonstrate multiple R2R iterations to mitigate different biases. Code is available on https://github.com/maxdreyer/Reveal2Revise.
Remaining Useful Life Prediction for Batteries Utilizing an Explainable AI Approach with a Predictive Application for Decision-Making
Nepal Engineering College, Pokhara University(博卡拉大学尼泊尔工程学院);University of Delaware(特拉华大学);Delft University of Technology(代尔夫特理工大学);The Hague University of Applied Science(海牙应用科学大学);Institute of Engineering, Pulchowk Campus, Tribhuvan University(特里布文大学工程学院普尔乔克校区)
Explainable AI meets Healthcare: A Study on Heart Disease Dataset
Pandit Deendayal Petroleum University(潘迪特·迪恩达亚尔石油大学);Confirm SFI Centre for Smart Manufacturing(Confirm SFI智能制造中心);Data Science Institute(数据科学研究所);National University of Ireland Galway(爱尔兰国立大学戈尔韦分校)
Evaluating RE Practices for Explainability: Synthesizing Insights from Daimler Truck into an Explainable RE Framework Proposal
TUM School of CIT, Technical University of Munich(慕尼黑技术大学 CIT 学院);Institute of Software Engineering, University of Stuttgart(斯图加特大学软件工程研究所);Department of Computer Science, Vrije Universiteit Amsterdam(阿姆斯特丹自由大学计算机科学系)
SCOPE-PD: Explainable AI on Subjective and Clinical Objective Measurements of Parkinson's Disease for Precision Decision-Making
Computing & Information Sciences(计算与信息科学);Florida International University(佛罗里达国际大学);Computer Science & Engineering(计算机科学与工程);University of South Florida(佛罗里达州立大学)
Ethical and Explainable AI in Reusable MLOps Pipelines
Washington University of Science and Technology(科学与技术华盛顿大学);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Karlstad University(卡尔斯塔德大学);American University of Bahrain(巴林美国大学)
Spoof Trace Discovery for Deep Learning Based Explainable Face Anti-Spoofing
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS);China Mobile Financial Technology Co., Ltd.(中国移动金融科技有限公司);Guangzhou Pixel Solutions Co., Ltd.(广州像素解决方案有限公司);Center for Machine Vision and Signal Analysis, University of Oulu(奥卢大学机器视觉与信号分析中心);CAIR, HKSIS, Chinese Academy of Sciences(中国科学院香港创新研究所有限公司);SCSE, the Faculty of Innovation Engineering, M.U.S.T(澳门创新工程学院SCSE)
SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imaging
Florida Institute of Technology(佛罗里达理工学院);Knovel Engineering Lab(诺维尔工程实验室);Vietnam Military Medical University(越南军事医科大学);Military Central Hospital(108陆军中央医院);Can Tho University of Medicine and Pharmacy(芹苴医药大学);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校)
Explainable Quantum Machine Learning for Multispectral Images Segmentation: Case Study
Systems Research Institute Polish Academy of Sciences(波兰科学院系统研究所);Nicolaus Copernicus Astronomical Center, Polish Academy of Sciences(波兰科学院尼古拉·哥白尼天文中心);Center of Excellence in Artificial Intelligence, AGH University(AGH大学人工智能卓越中心)
COVID-Net USPro: An Open-Source Explainable Few-Shot Deep Prototypical Network to Monitor and Detect COVID-19 Infection from Point-of-Care Ultrasound Images
University of Waterloo(滑铁卢大学);National Research Council Canada(加拿大国家研究委员会);McGill University(麦吉尔大学)