arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.02652cs.LGcs.AIstat.ML
收藏

通过强制条件单调性实现早退架构中的随时分类

Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity

UvA-Bosch Delta Lab(阿姆斯特丹大学-博世三角洲实验室);University of Amsterdam(阿姆斯特丹大学);University of Cambridge(剑桥大学);Bosch Center for AI(博世人工智能中心);University of Tübingen(蒂宾根大学)

Metod Jazbec, James Urquhart Allingham, Dan Zhang, Eric Nalisnick

AI总结:

本文针对早退网络在随时计算中预测质量不随计算时间单调提升的问题,提出基于专家乘积的事后修改方法,赋予模型条件单调性,在保持平均准确率的同时实现真正的随时预测。

摘要:

现代预测模型通常部署在计算预算动态变化的环境中。随时算法非常适合这类环境,因为在计算过程中的任意时刻,它们都能输出一个质量随计算时间变化的预测结果。早退神经网络因其能够在网络的各个阶段提供中间预测的能力,在随时计算的背景下引起了广泛关注。然而,我们证明当前的早退网络并不能直接适用于随时设置,因为对于单个数据点而言,预测质量并不能保证随着计算时间的延长而提升。为了解决这一不足,我们提出了一种基于专家乘积的优雅的事后修改方法,该方法促使早退网络逐渐变得更加自信。这使得我们的深度模型具备了预测质量的条件单调性——这是利用早退架构实现真正随时预测建模的重要基石。我们在标准图像分类任务上的实证结果表明,在保持平均竞争性准确率的同时,可以实现上述行为。

英文摘要:

Modern predictive models are often deployed to environments in which computational budgets are dynamic. Anytime algorithms are well-suited to such environments as, at any point during computation, they can output a prediction whose quality is a function of computation time. Early-exit neural networks have garnered attention in the context of anytime computation due to their capability to provide intermediate predictions at various stages throughout the network. However, we demonstrate that current early-exit networks are not directly applicable to anytime settings, as the quality of predictions for individual data points is not guaranteed to improve with longer computation. To address this shortcoming, we propose an elegant post-hoc modification, based on the Product-of-Experts, that encourages an early-exit network to become gradually confident. This gives our deep models the property of conditional monotonicity in the prediction quality -- an essential stepping stone towards truly anytime predictive modeling using early-exit architectures. Our empirical results on standard image-classification tasks demonstrate that such behaviors can be achieved while preserving competitive accuracy on average.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑