Always Keep Your Promises: A Model-Agnostic Attribution Algorithm for Neural Networks
David R. Cheriton School of Computer Science, University of Waterloo, ON, Canada(滑铁卢大学戴维·R·切里顿计算机科学学院);Scotiabank AML AI Research, Toronto, ON, Canada(多伦多加拿大Scotiabank反洗钱AI研究)
Kevin Lee, Duncan Smith-Halverson, Pablo Millan Arias
层间相关传播(LRP)通过守恒性质和深度泰勒分解的基础,为神经网络提供了一种原理化的属性分配方法。然而,现有实现是在模块层面进行的,需要特定架构的传播规则和模型修改。这些限制了目标模型的通用性和实现的可持续性,随着架构的发展而演变。我们引入了DynamicLRP,一种在张量操作层面运行的模型无关LRP框架。通过将属性分解到计算图中的单个操作,并引入一种名为Promise系统的新型机制来延迟激活解决,我们的方法实现了真正的架构无关性,同时保持LRP的理论保证。这种设计独立于反向传播机制,不需要模型修改,允许与梯度反向传播并行执行。基于计算图,这种方法在理论上可以扩展到其他支持自动微分的深度学习库。我们展示了与专门实现相匹配或超过(在VGG上1.77 vs 1.69 ABPC,等效性能在ViT上,93.70%和95.06%的top-1属性准确性在解释RoBERTa-large和Flan-T5-large的答案在SQuADv2上)同时在具有1亿至十亿参数的模型上保持实际效率。我们实现了99.92%的节点覆盖,覆盖了31,465个计算图节点,来自15种不同的架构,包括状态空间模型(Mamba)、音频转换器(Whisper)和多模态系统(DePlot),没有任何特定模型的代码或规则,针对47种基本操作进行了实现。我们的操作层面分解和Promise系统建立了LRP在不断演变的架构中的可持续、可扩展的基础。所有代码可在https://github.com/keeinlev/dynamicLRP上获得。
英文摘要:
Layer-wise Relevance Propagation (LRP) provides principled attribution for neural networks through conservation properties and foundations in Deep Taylor Decomposition. However, existing implementations operate at the module level, requiring architecture-specific propagation rules and model modifications. These limit the generality of target model and sustainability of implementations as architectures evolve. We introduce DynamicLRP, a model-agnostic LRP framework operating at the tensor operation level. By decomposing attribution to individual operations within computation graphs and introducing a novel mechanism for deferred activation resolution, named the Promise System, our approach achieves true architecture agnosticity while maintaining LRP's theoretical guarantees. This design operates independently of backpropagation machinery, requiring no model modification, enabling side-by-side execution with gradient backpropagation. Being based on computation graphs, this method is theoretically extensible to other deep learning libraries that support auto-differentiation. We demonstrate faithfulness matching or exceeding specialized implementations (1.77 vs 1.69 ABPC on VGG, equivalent performance on ViT, 93.70% and 95.06% top-1 attribution accuracy for explaining RoBERTa-large and Flan-T5-large answers on SQuADv2, respectively) while maintaining practical efficiency on models with 100M-1B parameters. We achieved 99.92% node coverage across 31,465 computation graph nodes from 15 diverse architectures, including state-space models (Mamba), audio transformers (Whisper), and multimodal systems (DePlot) without any model-specific code with rules for 47 fundamental operations implemented. Our operation-level decomposition and Promise System establish a sustainable, extensible foundation for LRP across evolving architectures. All code is available at https://github.com/keeinlev/dynamicLRP .
School of Computer Science, Peking University, Beijing 100871, China(北京大学计算机科学学院);Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education(高可信软件技术重点实验室(北京大学))
Who Does What in Deep Learning? Multidimensional Game-Theoretic Attribution of Function of Neural Units
Institute of Computational Neuroscience(计算神经科学研究所);University Medical Center Eppendorf(埃本多夫大学医学中心);International Max Planck Research School on Cognitive Neuroimaging(认知神经成像国际马克斯·普朗克研究学校);MRC Cognition and Brain Sciences Unit(MRC认知与脑科学单位);Mila - Quebec Artificial Intelligence Institute(魁北克人工智能研究所);Learning in Machines & Brains(机器与大脑学习);Departments of Bioengineering and Neuroscience(生物工程与神经科学系);Department of Health Sciences(健康科学系)
Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property
STAIR Lab, Chiba Institute of Technology(千叶技术学院STAIR实验室);School of Mathematics and Statistics, The University of Melbourne(墨尔本大学数学与统计学学院);ZOZO Research(ZOZO研究)
SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
Zhejiang University(浙江大学);Ant Digital Technologies, Ant Group(蚂蚁集团数字技术部);Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph(浙江大学-蚂蚁集团知识图谱联合实验室)