分数跳跃：面向更细粒度的动态CNN推理
Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
摘要：
虽然越来越深的网络仍然是实现最先进性能的普遍需求，但对于许多特定输入，更简单的网络可能已经足够了。现有的工作通过学习以依赖于输入的方式跳过卷积层来利用这种观察。然而，我们认为他们的二元决策方案，即，完全执行或完全绕过用于特定输入的一个层，可以通过引入更细粒度的、“更软”的决策来增强。因此，我们提出了一个动态分数跳过（DFS）框架。DFS的核心思想是假设逐层量化（到不同的位宽）作为在充分利用和跳过层之间做出的中间“软”选择。对于每个输入，DFS动态地将位宽分配给每个层的权重和激活，其中完全执行和跳过可以被视为两个“极端”（即，全位宽和零位宽）。通过这种方式，DFS可以在输入自适应推理期间“部分地”利用层的表达能力，从而实现更细粒度的准确性-计算成本权衡。它提出了一个统一的视图链接输入自适应层跳过和输入自适应混合量化。大量的实验结果表明，优越的计算成本和模型的表达能力（准确性）之间的权衡实现DFS。更多的可视化也表明DFS行为中的平滑和一致的过渡，特别是当总计算预算变化时，层跳过和不同量化之间的学习选择，验证了我们的假设，即层量化可以被视为层跳过的中间变体。我们的源代码和补充材料可在\link{https://github.com/Torment123/DFS}上获得。
英文摘要：
While increasingly deep networks are still in general desired for achieving state-of-the-art performance, for many specific inputs a simpler network might already suffice. Existing works exploited this observation by learning to skip convolutional layers in an input-dependent manner. However, we argue their binary decision scheme, i.e., either fully executing or completely bypassing one layer for a specific input, can be enhanced by introducing finer-grained, "softer" decisions. We therefore propose a Dynamic Fractional Skipping (DFS) framework. The core idea of DFS is to hypothesize layer-wise quantization (to different bitwidths) as intermediate "soft" choices to be made between fully utilizing and skipping a layer. For each input, DFS dynamically assigns a bitwidth to both weights and activations of each layer, where fully executing and skipping could be viewed as two "extremes" (i.e., full bitwidth and zero bitwidth). In this way, DFS can "fractionally" exploit a layer's expressive power during input-adaptive inference, enabling finer-grained accuracy-computational cost trade-offs. It presents a unified view to link input-adaptive layer skipping and input-adaptive hybrid quantization. Extensive experimental results demonstrate the superior tradeoff between computational cost and model expressive power (accuracy) achieved by DFS. More visualizations also indicate a smooth and consistent transition in the DFS behaviors, especially the learned choices between layer skipping and different quantizations when the total computational budgets vary, validating our hypothesis that layer quantization could be viewed as intermediate variants of layer skipping. Our source code and supplementary material are available at \link{https://github.com/Torment123/DFS}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
