arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.04369cs.LGstat.ML
收藏

多头神经网络搜索

Multi-headed Neural Ensemble Search

University of Freiburg(弗赖堡大学);Bosch Center for Artificial Intelligence(博世人工智能中心)

Ashwin Raaghav Narayanan, Arber Zela, Tonmoy Saikia, Thomas Brox, Frank Hutter

摘要:

已知使用不同种子训练的CNN模型的集合(也称为深度集合)实现优于CNN的单个副本的性能。神经网络搜索(NES)可以通过增加架构多样性来进一步提高性能。然而,在有限的计算资源下,NES的范围仍然令人望而却步。在这项工作中,我们将NES扩展到多头合奏,其中包括一个共享的骨干连接到多个预测头。与Deep Ensemble不同,这些多头集成可以端到端训练,这使我们能够利用一次性NAS方法来优化集成目标。通过广泛的实证评估,我们证明了多头集成搜索找到强大的集成快3倍,同时具有可比的性能,其他集成搜索方法,在预测性能和不确定性校准。

英文摘要:

Ensembles of CNN models trained with different seeds (also known as Deep Ensembles) are known to achieve superior performance over a single copy of the CNN. Neural Ensemble Search (NES) can further boost performance by adding architectural diversity. However, the scope of NES remains prohibitive under limited computational resources. In this work, we extend NES to multi-headed ensembles, which consist of a shared backbone attached to multiple prediction heads. Unlike Deep Ensembles, these multi-headed ensembles can be trained end to end, which enables us to leverage one-shot NAS methods to optimize an ensemble objective. With extensive empirical evaluations, we demonstrate that multi-headed ensemble search finds robust ensembles 3 times faster, while having comparable performance to other ensemble search methods, in both predictive performance and uncertainty calibration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑