AI 中文总结
本研究通过交互驱动分析，发现参数剪枝对DNN性能的影响取决于是否移除可泛化的低阶交互模式，并揭示了三阶段动态规律。
AI 中文摘要
本研究聚焦于理解当不同参数被剪枝时，深度神经网络（DNN）性能出现多样化退化的内在因素这一科学问题。为了解释为何剪枝某些参数会导致显著的性能退化，而剪枝其他参数却不会，我们考察了剪枝操作如何影响DNN编码的交互模式。我们发现，当逐步增加剪枝比例时，DNN编码的交互模式呈现出明显的三阶段动态，即模型性能在剪枝操作开始移除低阶交互之前不会受到较大影响，且低阶交互表现出很强的泛化能力。此外，我们发现DNN性能对某些模块剪枝的高敏感性归因于剪枝操作是否移除了可泛化的低阶交互模式。
英文摘要
This study focuses on the scientific problem of understanding internal factors that govern the diverse performance degradation of deep neural networks (DNNs) when different parameters are pruned. In order to explain why pruning certain parameters leads to significant performance degradation but pruning other parameters does not, we examine how the pruning operation affects the interaction patterns encoded by the DNN. We find that when we progressively increase the pruning ratio, the interaction patterns encoded by DNNs exhibit a distinct three-phase dynamics, \emph{i.e.}, model performance is not largely affected until the pruning operation begins to remove low-order interactions, and low-order interactions exhibit strong generalizability. Moreover, we find that the high sensitivity of DNN performance to the pruning of certain modules is attributed to whether the pruning operation removes generalizable low-order interaction patterns.
