arXiv 2310.199092023-11-20cs.CVcs.LG

骨干网络之战：跨计算机视觉任务的预训练模型大规模比较

Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks

New York University（纽约大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Maryland（马里兰大学）；Georgia Institute of Technology（佐治亚理工学院）；Inria（法国国家信息与自动化研究所）；Meta AI Research（元人工智能研究部门）

Micah Goldblum, Hossein Souri, Renkun Ni, Manli Shu, Viraj Prabhu, Gowthami Somepalli, Prithvijit Chattopadhyay, Mark Ibrahim, Adrien Bardes, Judy Hoffman, Rama… 展开作者

Micah Goldblum, Hossein Souri, Renkun Ni, Manli Shu, Viraj Prabhu, Gowthami Somepalli, Prithvijit Chattopadhyay, Mark Ibrahim, Adrien Bardes, Judy Hoffman, Rama Chellappa, Andrew Gordon Wilson, Tom Goldstein

备注：: Accepted to NeurIPS 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Battle of the Backbones (BoB)基准，跨多种计算机视觉任务比较1500+次训练运行的预训练模型，发现监督预训练CNN在多数任务表现最佳，而SSL骨干网络在公平比较中极具竞争力。

摘要：

基于神经网络的计算机视觉系统通常构建在骨干网络（backbone）之上，这是一种预训练或随机初始化的特征提取器。几年前，默认选择是在ImageNet上训练的卷积神经网络。然而，近年来涌现出无数使用各种算法和数据集预训练的骨干网络。虽然这种丰富的选择为一系列系统带来了性能提升，但从业者很难做出明智的决策来选择合适的骨干网络。Battle of the Backbones (BoB) 通过对多种预训练模型进行基准测试，使这一选择变得更容易，这些模型包括视觉-语言模型、通过自监督学习训练的模型以及Stable Diffusion骨干网络，测试任务涵盖从分类到目标检测再到OOD泛化等多种计算机视觉任务。此外，BoB通过对超过1500次训练运行进行全面分析，阐明了现有方法的优缺点，为研究社区推进计算机视觉指明了有前景的方向。尽管视觉Transformer (ViTs) 和自监督学习 (SSL) 越来越受欢迎，我们发现，在我们考虑的模型中，在大型训练集上以监督方式预训练的卷积神经网络在大多数任务上仍然表现最佳。此外，在相同架构和相似大小预训练数据集的公平比较中，我们发现SSL骨干网络极具竞争力，这表明未来的工作应使用先进架构和更大的预训练数据集进行SSL预训练。我们在此发布了实验的原始结果以及允许研究人员将自己的骨干网络通过严苛测试的代码：https://github.com/hsouri/Battle-of-the-Backbones

英文摘要：

Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an ImageNet-trained convolutional neural network. However, the recent past has seen the emergence of countless backbones pretrained using various algorithms and datasets. While this abundance of choice has led to performance increases for a range of systems, it is difficult for practitioners to make informed decisions about which backbone to choose. Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more. Furthermore, BoB sheds light on promising directions for the research community to advance computer vision by illuminating strengths and weakness of existing approaches through a comprehensive analysis conducted on more than 1500 training runs. While vision transformers (ViTs) and self-supervised learning (SSL) are increasingly popular, we find that convolutional neural networks pretrained in a supervised fashion on large training sets still perform best on most tasks among the models we consider. Moreover, in apples-to-apples comparisons on the same architectures and similarly sized pretraining datasets, we find that SSL backbones are highly competitive, indicating that future works should perform SSL pretraining with advanced architectures and larger pretraining datasets. We release the raw results of our experiments along with code that allows researchers to put their own backbones through the gauntlet here: https://github.com/hsouri/Battle-of-the-Backbones

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。