arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.04849cs.LGcs.CVstat.ML
收藏

动态路由网络

Dynamic Routing Networks

National University of Singapore(新加坡国立大学)

Shaofeng Cai, Yao Shu, Wei Wang, Beng Chin Ooi

摘要:

深度神经网络在现实应用中的部署主要受到其高推理成本的限制。人们已经做出了大量的努力来提高专家设计或算法搜索架构的准确性。然而,增量改进通常是通过越来越昂贵的模型来实现的,只有一小部分输入实例真正需要这些模型。因此,使用通过相同的转换处理所有输入实例的静态架构进行推断将招致不必要的计算成本。因此,以实例感知的方式定制模型容量对于更高的推理效率是非常必要的。在本文中,我们提出了动态路由网络(DRNets),它支持有效的实例感知推理,通过路由输入实例,只有必要的转换分支从一个候选集的分支转换节点之间的每个连接。分支选择是通过相应的分支重要性权重动态确定的,这些权重首先从轻量级超网络(RouterNets)中生成,然后在选择之前用Gumbel-Softmax重新校准。大量的实验表明,DRNets可以在推理过程中减少大量的参数大小和FLOP,其预测性能与最先进的架构相当。

英文摘要:

The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs. Extensive efforts have been made to improve the accuracy with expert-designed or algorithm-searched architectures. However, the incremental improvement is typically achieved with increasingly more expensive models that only a small portion of input instances really need. Inference with a static architecture that processes all input instances via the same transformation would thus incur unnecessary computational costs. Therefore, customizing the model capacity in an instance-aware manner is much needed for higher inference efficiency. In this paper, we propose Dynamic Routing Networks (DRNets), which support efficient instance-aware inference by routing the input instance to only necessary transformation branches selected from a candidate set of branches for each connection between transformation nodes. The branch selection is dynamically determined via the corresponding branch importance weights, which are first generated from lightweight hypernetworks (RouterNets) and then recalibrated with Gumbel-Softmax before the selection. Extensive experiments show that DRNets can reduce a substantial amount of parameter size and FLOPs during inference with prediction performance comparable to state-of-the-art architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑