arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.06580cs.LG
收藏

SAE:单架构集成神经网络

SAE: Single Architecture Ensemble Neural Networks

University College London(伦敦大学学院);Imperial College London(帝国理工学院)

Martin Ferianc, Hongxiang Fan, Miguel Rodrigues

AI总结:

提出SAE框架,通过可扩展搜索空间和优化目标自动联合搜索单网络内的早期退出与多输入多输出配置,在保持准确度与置信度校准的同时显著降低计算操作或参数量。

摘要:

独立神经网络(NN)的集成在各类任务中表现出比单个NN更高的准确性和置信度校准。为提高独立NN集成的硬件效率,近期方法通过添加早期退出或考虑多输入多输出方法在单个网络内构建集成。然而,尚不清楚哪种方法对给定任务最有效,需要手动且分别搜索每种方法。我们新颖的单架构集成(SAE)框架能够自动联合搜索早期退出和多输入多输出配置及其此前未观察到的中间组合。SAE包含两部分:一个可扩展的搜索空间,泛化了先前方法及其中间配置;以及一个优化目标,允许为给定任务学习最优配置。我们的图像分类和回归实验表明,使用SAE我们可以自动找到适合任务的多样化配置,实现与基线相当的准确度或置信度校准,同时将计算操作或参数量减少$1.5{\sim}3.7\times$。

英文摘要:

Ensembles of separate neural networks (NNs) have shown superior accuracy and confidence calibration over single NN across tasks. To improve the hardware efficiency of ensembles of separate NNs, recent methods create ensembles within a single network via adding early exits or considering multi input multi output approaches. However, it is unclear which of these methods is the most effective for a given task, needing a manual and separate search through each method. Our novel Single Architecture Ensemble (SAE) framework enables an automatic and joint search through the early exit and multi input multi output configurations and their previously unobserved in-between combinations. SAE consists of two parts: a scalable search space that generalises the previous methods and their in-between configurations, and an optimisation objective that allows learning the optimal configuration for a given task. Our image classification and regression experiments show that with SAE we can automatically find diverse configurations that fit the task, achieving competitive accuracy or confidence calibration to baselines while reducing the compute operations or parameter count by up to $1.5{\sim}3.7\times$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑