arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.05343cs.CVcs.ITcs.LGmath.IT
收藏

利用编码 ResNeXt 解耦信息路径

Towards Disentangling Information Paths with Coded ResNeXt

EURECOM(欧洲通信学院);Amadeus(艾玛迪斯)

Apostolos Avranas, Marios Kountouris

AI总结:

本文提出一种基于编码理论预先设计类别信息路径的 ResNeXt 分类架构,使网络具备可解释性、轻量级二分类器提取、中间层早期预测能力,并同时提升整体准确率。

摘要:

传统且广泛使用的深度学习模型黑箱处理方法对引导神经网络决策的机制提供的洞见有限或几乎没有。大量研究工作致力于构建可解释模型以解决这一问题。大多数工作要么聚焦于与最后几层相关的高层特征,要么试图解释单个层的输出。在本文中,我们采用一种新颖方法来增强整个网络功能的透明度。我们提出了一种用于分类的神经网络架构,其中与每个类别相关的信息沿着特定路径流动。这些路径在训练之前借助编码理论预先设计,并且不依赖于类别之间的语义相似性。一个关键特性是每条路径都可以作为一个自主的单一用途模型使用。这使我们能够在没有任何额外训练的情况下,针对任意类别获得一个轻量级二分类器,其参数数量比原始网络至少少 60%。此外,我们基于编码理论的方法允许神经网络在推理过程中于中间层进行早期预测,而无需完整评估整个网络。值得注意的是,所提出的架构在提供上述所有特性的同时,还提高了整体准确率。我们在 CIFAR-10/100 和 ImageNet-1k 上测试的略微修改的 ResNeXt 模型上展示了这些特性。

英文摘要:

The conventional, widely used treatment of deep learning models as black boxes provides limited or no insights into the mechanisms that guide neural network decisions. Significant research effort has been dedicated to building interpretable models to address this issue. Most efforts either focus on the high-level features associated with the last layers, or attempt to interpret the output of a single layer. In this paper, we take a novel approach to enhance the transparency of the function of the whole network. We propose a neural network architecture for classification, in which the information that is relevant to each class flows through specific paths. These paths are designed in advance before training leveraging coding theory and without depending on the semantic similarities between classes. A key property is that each path can be used as an autonomous single-purpose model. This enables us to obtain, without any additional training and for any class, a lightweight binary classifier that has at least $60\%$ fewer parameters than the original network. Furthermore, our coding theory based approach allows the neural network to make early predictions at intermediate layers during inference, without requiring its full evaluation. Remarkably, the proposed architecture provides all the aforementioned properties while improving the overall accuracy. We demonstrate these properties on a slightly modified ResNeXt model tested on CIFAR-10/100 and ImageNet-1k.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑