LayerShuffle：通过随机化层执行顺序增强视觉Transformer的鲁棒性
LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order
AI总结：
针对人工神经网络测试时对层剪枝或打乱操作鲁棒性不足的问题，提出LayerShuffle方法，通过训练时随机化视觉Transformer注意力模块执行顺序，使其测试时适应任意层序（精度降约20%），且剪枝后性能优雅下降。
摘要：
由于其架构和训练方式，人工神经网络在测试时通常对剪枝或层打乱操作不具备鲁棒性。然而，这类特性在不同应用中十分必要，例如分布式神经网络架构（执行顺序无法保证）或推理时部分网络可能失效的场景。本研究通过多种训练方法解决这些问题，针对视觉Transformer，其核心组件是在训练时随机化注意力模块的执行顺序。通过所提方法，视觉Transformer在测试时能够适应任意层执行顺序，前提是容忍相同模型大小下约20%的精度下降。我们分析了训练后模型的特征表示，以及推理时每层如何根据其位置对模型预测做出贡献。分析表明，层会根据其在网络中的位置学习不同的贡献方式。最后，我们在测试时对模型进行层剪枝，发现其性能优雅下降。代码可在https://github.com/matfrei/layershuffle获取。
英文摘要：
Due to their architecture and how they are trained, artificial neural networks are typically not robust toward pruning or shuffling layers at test time. However, such properties would be desirable for different applications, such as distributed neural network architectures where the order of execution cannot be guaranteed or parts of the network can fail during inference. In this work, we address these issues through a number of training approaches for vision transformers whose most important component is randomizing the execution order of attention modules at training time. With our proposed approaches, vision transformers are capable to adapt to arbitrary layer execution orders at test time assuming one tolerates a reduction (about 20\%) in accuracy at the same model size. We analyse the feature representations of our trained models as well as how each layer contributes to the models prediction based on its position during inference. Our analysis shows that layers learn to contribute differently based on their position in the network. Finally, we layer-prune our models at test time and find that their performance declines gracefully. Code available at https://github.com/matfrei/layershuffle.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
