arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.01474cs.CV
收藏

基于Distilled Hierarchical Neural Ensembles的任意时间推理

Anytime Inference with Distilled Hierarchical Neural Ensembles

Facebook AI Research(脸书人工智能研究)

Adria Ruiz, Jakob Verbeek

摘要:

深度神经网络中的推理在计算上可能是昂贵的,并且能够随时进行推理的网络在计算量或输入数据的量随时间变化的mscenario中是重要的。在这样的网络中,推理过程可以中断以更快地提供结果,或者继续以获得更准确的结果。我们提出了分层神经集成(HNE),一种新的框架,嵌入一个集成的多个网络在一个分层的树结构,共享中间层。在HNE中,我们通过评估集合中更多或更少的模型来控制推理的复杂性。我们的第二个贡献是一个新的分层蒸馏方法,以提高预测精度的小合奏。这种方法利用了我们集合的嵌套结构,在各个模型之间优化分配准确性和多样性。我们的实验表明,与以前的任何时间推断模型相比,HNE在CIFAR-10/100和ImageNet数据集上提供了最先进的准确性-计算权衡。

英文摘要:

Inference in deep neural networks can be computationally expensive, and networks capable of anytime inference are important in mscenarios where the amount of compute or quantity of input data varies over time. In such networks the inference process can interrupted to provide a result faster, or continued to obtain a more accurate result. We propose Hierarchical Neural Ensembles (HNE), a novel framework to embed an ensemble of multiple networks in a hierarchical tree structure, sharing intermediate layers. In HNE we control the complexity of inference on-the-fly by evaluating more or less models in the ensemble. Our second contribution is a novel hierarchical distillation method to boost the prediction accuracy of small ensembles. This approach leverages the nested structure of our ensembles, to optimally allocate accuracy and diversity across the individual models. Our experiments show that, compared to previous anytime inference models, HNE provides state-of-the-art accuracy-computate trade-offs on the CIFAR-10/100 and ImageNet datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑