双动态推理：实现更高效、自适应且可控的深度推理
Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference
AI总结：
针对卷积神经网络推理能耗高难以部署于资源受限物联网的问题，提出整合输入与资源依赖机制、支持多粒度跳过的双动态推理框架，可适配多种骨干网络与资源目标，实现更优且可控的精度-资源权衡。
摘要：
当前最先进的卷积神经网络（CNN）取得了破纪录的预测性能，但代价是推理过程能耗极高，这阻碍了它们在资源受限的物联网（IoT）应用中的广泛部署。我们提出了双动态推理（DDI）框架，核心特点如下：1）我们将输入依赖型和资源依赖型的动态推理机制整合到统一框架下，以适配实际中多变的物联网资源需求。DDI既能持续降低简单样本的不必要计算开销，也能为满足硬性资源约束终止所有样本的推理；2）针对输入依赖型推理，我们提出灵活的多粒度学习跳过（MGL2S）方法，支持同时进行层级和通道级的跳过操作；3）我们将DDI扩展至DenseNet等复杂CNN骨干网络，并证明DDI可用于优化任意特定资源目标，包括推理延迟或能耗。大量实验表明，与现有同类方法相比，DDI实现了更优的推理精度-资源权衡，且调控该权衡的灵活性更强。具体而言，与现有主流基线方法相比，在精度持平甚至更高的情况下，DDI最高可节省4倍计算量。
英文摘要：
State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications. We propose a dual dynamic inference (DDI) framework that highlights the following aspects: 1) we integrate both input-dependent and resource-dependent dynamic inference mechanisms under a unified framework in order to fit the varying IoT resource requirements in practice. DDI is able to both constantly suppress unnecessary costs for easy samples, and to halt inference for all samples to meet hard resource constraints enforced; 2) we propose a flexible multi-grained learning to skip (MGL2S) approach for input-dependent inference which allows simultaneous layer-wise and channel-wise skipping; 3) we extend DDI to complex CNN backbones such as DenseNet and show that DDI can be applied towards optimizing any specific resource goals including inference latency or energy cost. Extensive experiments demonstrate the superior inference accuracy-resource trade-off achieved by DDI, as well as the flexibility to control such trade-offs compared to existing peer methods. Specifically, DDI can achieve up to 4 times computational savings with the same or even higher accuracy as compared to existing competitive baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
