通过垂直切片网络进行随时神经预测
Anytime Neural Prediction via Slicing Networks Vertically
摘要：
先驱深度神经网络（DNN）已经出现更深或更宽，以提高其在各种人工智能应用中的准确性。然而，DNN通常太重而无法在实践中部署，并且通常需要在给定计算资源预算的情况下动态地控制它们的架构，即，随时预测虽然大多数现有的方法都集中在联合训练多个浅层子网络，但我们研究训练薄子网络。为此，我们首先在对现有多分支DNN进行微小修改的情况下构建了许多包含性的瘦子网络（具有相同的深度），并发现它们可以显著优于最先进的密集架构，用于随时预测。由于它们的简单性和有效性，这是显着的，但是联合训练许多瘦子网络面临着训练复杂性的新挑战。为了解决这个问题，我们还提出了一种新的DNN架构，通过对多分支网络参数强制某种稀疏模式，使它们有效地进行训练，以实现随时预测的目的。在我们对ImageNet数据集的实验中，与最先进的任何时间模型相比，其子网络具有高达$43.3\%$更小的大小（FLOP），并且具有相同的精度。最后，我们还提出了一个替代的任务下提出的架构，使用层次分类法，这带来了一个新的角度随时预测。
英文摘要：
The pioneer deep neural networks (DNNs) have emerged to be deeper or wider for improving their accuracy in various applications of artificial intelligence. However, DNNs are often too heavy to deploy in practice, and it is often required to control their architectures dynamically given computing resource budget, i.e., anytime prediction. While most existing approaches have focused on training multiple shallow sub-networks jointly, we study training thin sub-networks instead. To this end, we first build many inclusive thin sub-networks (of the same depth) under a minor modification of existing multi-branch DNNs, and found that they can significantly outperform the state-of-art dense architecture for anytime prediction. This is remarkable due to their simplicity and effectiveness, but training many thin sub-networks jointly faces a new challenge on training complexity. To address the issue, we also propose a novel DNN architecture by forcing a certain sparsity pattern on multi-branch network parameters, making them train efficiently for the purpose of anytime prediction. In our experiments on the ImageNet dataset, its sub-networks have up to $43.3\%$ smaller sizes (FLOPs) compared to those of the state-of-art anytime model with respect to the same accuracy. Finally, we also propose an alternative task under the proposed architecture using a hierarchical taxonomy, which brings a new angle for anytime prediction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
