发表机构
LTCI, Télécom Paris, Institut Polytechnique de Paris; University of Turin（LTCI，巴黎电信学院，巴黎综合理工学院; 都灵大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对深度神经网络压缩中层合并方法的局限，提出无解析解也可合并层且不增大核大小的高效策略，在多架构、多数据集及嵌入式平台验证了其推理加速效果。
AI 中文摘要
尽管深度神经网络已成为机器学习诸多领域的基础，但高计算需求限制了其在资源受限环境中的应用。为解决该问题，深度压缩方法被提出以识别并线性化冗余激活函数，从而实现无中间非线性的层合并。然而，这些方法面临两大关键挑战：因缺乏带填充卷积层合并的解析解，无法直接应用于带填充的卷积；且通常会增大合并层的核大小，进而限制加速增益。为克服这些局限，本文提出一种高效策略，可实现无现有解析解的层合并，且不会增大核大小。我们在多种架构与数据集上验证了该方法，并在真实嵌入式平台上测量了推理加速增益，已公开代码至指定网址。
英文摘要
Although Deep Neural Networks have become foundational in many areas of Machine Learning, high computational demands limit their application in resource-constrained environments. To address this issue, depth compression methods have been proposed to identify and linearize redundant activation functions, thereby allowing for the merging of layers without intermediate non-linearities. However, these methods face two key challenges: they cannot be directly applied to convolutions with padding due to the absence of an analytical solution for merging these layers, and they typically increase the kernel size of merged layers, thus limiting speed-up gains. To overcome these limitations, we propose an efficient strategy that enables merging of layers without an existing analytical solution, and also without increasing kernel size. We validate our approach across multiple architectures and datasets, and measure inference speed-up gains on real embedded platforms. We publicly released the code at https://github.com/ShulzhenkoPetr/deep-to-shallow.
Comments13 pages, 3 figures, accepted at the ITEM Workshop at ECML PKDD 2026
