arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.05409cs.LG
收藏

Zero Time Waste:提前退出神经网络中的循环预测

Zero Time Waste: Recycling Predictions in Early Exit Neural Networks

Jagiellonian University(雅盖隆大学);Warsaw University of Technology(华沙理工大学);Tooploox

Maciej Wołczyk, Bartosz Wójcik, Klaudia Bałazy, Igor Podolak, Jacek Tabor, Marek Śmieja, Tomasz Trzciński

摘要:

减少大型深度学习模型的处理时间的问题是许多现实世界应用中的基本挑战。早期退出方法通过将额外的内部分类器(IC)附加到神经网络的中间层来努力实现这一目标。IC可以快速返回简单示例的预测,从而减少整个模型的平均推理时间。然而,如果一个特定的IC没有决定提前返回答案,它的预测就会被丢弃,计算就会被浪费掉。为了解决这个问题,我们引入了零时间浪费(ZTW),这是一种新的方法,其中每个IC通过(1)在IC之间添加直接连接以及(2)以类似集成的方式组合先前的输出来重用其前任返回的预测。我们在各种数据集和架构上进行了广泛的实验,以证明ZTW比其他最近提出的早期退出方法实现了更好的准确性与推理时间的权衡。

英文摘要:

The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by attaching additional Internal Classifiers (ICs) to intermediate layers of a neural network. ICs can quickly return predictions for easy examples and, as a result, reduce the average inference time of the whole model. However, if a particular IC does not decide to return an answer early, its predictions are discarded, with its computations effectively being wasted. To solve this issue, we introduce Zero Time Waste (ZTW), a novel approach in which each IC reuses predictions returned by its predecessors by (1) adding direct connections between ICs and (2) combining previous outputs in an ensemble-like manner. We conduct extensive experiments across various datasets and architectures to demonstrate that ZTW achieves a significantly better accuracy vs. inference time trade-off than other recently proposed early exit methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑