arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.07010cs.LG
收藏

始终信守承诺:一种针对神经网络的模型无关属性算法

Always Keep Your Promises: A Model-Agnostic Attribution Algorithm for Neural Networks

David R. Cheriton School of Computer Science, University of Waterloo, ON, Canada(滑铁卢大学戴维·R·切里顿计算机科学学院);Scotiabank AML AI Research, Toronto, ON, Canada(多伦多加拿大Scotiabank反洗钱AI研究)

Kevin Lee, Duncan Smith-Halverson, Pablo Millan Arias

AI总结:

本文提出DynamicLRP,一种在张量操作层面运行的模型无关LRP框架,通过分解计算图中的属性并引入新的延迟激活解决机制,实现了真正的架构无关性,同时保持了LRP的理论保证,并在多个模型上展示了高准确性与效率。

摘要:

层间相关传播(LRP)通过守恒性质和深度泰勒分解的基础,为神经网络提供了一种原理化的属性分配方法。然而,现有实现是在模块层面进行的,需要特定架构的传播规则和模型修改。这些限制了目标模型的通用性和实现的可持续性,随着架构的发展而演变。我们引入了DynamicLRP,一种在张量操作层面运行的模型无关LRP框架。通过将属性分解到计算图中的单个操作,并引入一种名为Promise系统的新型机制来延迟激活解决,我们的方法实现了真正的架构无关性,同时保持LRP的理论保证。这种设计独立于反向传播机制,不需要模型修改,允许与梯度反向传播并行执行。基于计算图,这种方法在理论上可以扩展到其他支持自动微分的深度学习库。我们展示了与专门实现相匹配或超过(在VGG上1.77 vs 1.69 ABPC,等效性能在ViT上,93.70%和95.06%的top-1属性准确性在解释RoBERTa-large和Flan-T5-large的答案在SQuADv2上)同时在具有1亿至十亿参数的模型上保持实际效率。我们实现了99.92%的节点覆盖,覆盖了31,465个计算图节点,来自15种不同的架构,包括状态空间模型(Mamba)、音频转换器(Whisper)和多模态系统(DePlot),没有任何特定模型的代码或规则,针对47种基本操作进行了实现。我们的操作层面分解和Promise系统建立了LRP在不断演变的架构中的可持续、可扩展的基础。所有代码可在https://github.com/keeinlev/dynamicLRP上获得。

英文摘要:

Layer-wise Relevance Propagation (LRP) provides principled attribution for neural networks through conservation properties and foundations in Deep Taylor Decomposition. However, existing implementations operate at the module level, requiring architecture-specific propagation rules and model modifications. These limit the generality of target model and sustainability of implementations as architectures evolve. We introduce DynamicLRP, a model-agnostic LRP framework operating at the tensor operation level. By decomposing attribution to individual operations within computation graphs and introducing a novel mechanism for deferred activation resolution, named the Promise System, our approach achieves true architecture agnosticity while maintaining LRP's theoretical guarantees. This design operates independently of backpropagation machinery, requiring no model modification, enabling side-by-side execution with gradient backpropagation. Being based on computation graphs, this method is theoretically extensible to other deep learning libraries that support auto-differentiation. We demonstrate faithfulness matching or exceeding specialized implementations (1.77 vs 1.69 ABPC on VGG, equivalent performance on ViT, 93.70% and 95.06% top-1 attribution accuracy for explaining RoBERTa-large and Flan-T5-large answers on SQuADv2, respectively) while maintaining practical efficiency on models with 100M-1B parameters. We achieved 99.92% node coverage across 31,465 computation graph nodes from 15 diverse architectures, including state-space models (Mamba), audio transformers (Whisper), and multimodal systems (DePlot) without any model-specific code with rules for 47 fundamental operations implemented. Our operation-level decomposition and Promise System establish a sustainable, extensible foundation for LRP across evolving architectures. All code is available at https://github.com/keeinlev/dynamicLRP .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑