Recent neural architecture search (NAS) works proposed training-free metrics to rank networks which largely reduced the search cost in NAS. In this paper, we revisit these training-free metrics and find that: (1) the number of parameters (\#Param), which is the most straightforward training-free metric, is overlooked in previous works but is surprisingly effective, (2) recent training-free metrics largely rely on the \#Param information to rank networks. Our experiments show that the performance of recent training-free metrics drops dramatically when the \#Param information is not available. Motivated by these observations, we argue that metrics less correlated with the \#Param are desired to provide additional information for NAS. We propose a light-weight training-based metric which has a weak correlation with the \#Param while achieving better performance than training-free metrics at a lower search cost. Specifically, on DARTS search space, our method completes searching directly on ImageNet in only 2.6 GPU hours and achieves a top-1/top-5 error rate of 24.1\%/7.1\%, which is competitive among state-of-the-art NAS methods. Codes are available at \url{https://github.com/taoyang1122/Revisit_TrainingFree_NAS}
Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics
Xidian University(西安电子科技大学);Northwestern Polytechnical University(西北工业大学);China University of Mining Technology-Beijing(中国矿业大学(北京));Zhejiang Lab(之江实验室)
SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS
School of Computing Technologies, RMIT University(RMIT大学计算技术学院);University of Technology Sydney(悉尼科技大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
Mila, Université de Montréal(蒙特利尔大学 Mila 研究院);HCI/IWR, Heidelberg University(海德堡大学 HCI/IWR);University of Toronto(多伦多大学);The Alan Turing Institute, University of Oxford(牛津大学艾伦·图灵研究所)
Revisiting Metric Learning for Few-Shot Image Classification
The Chinese University of Hong Kong(香港中文大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Tencent AI Lab(腾讯人工智能实验室)
Revisiting Birds Eye View Perception Models with Frozen Foundation Models: DINOv2 and Metric3Dv2
University of Limerick(利默里克大学);SFI CRT Foundations in Data Science(爱尔兰科学基金会数据科学基础合作研究中心);Data Driven Computer Engineering (D²iCE) Research Centre(数据驱动计算机工程研究中心);Valeo Vision Systems(法雷奥视觉系统公司)
Revisiting the Evaluation of Deep Neural Networks for Pedestrian Detection
Stellantis, Opel Automobile GmbH(斯特拉蒂斯公司);Carl von Ossietzky Universität Oldenburg(卡尔·冯·奥西特茨基大学);Deutsches Zentrum für Luft- und Raumfahrt(德国航空航天中心);Universität Ulm(乌尔姆大学)
Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union
KU Leuven(鲁汶大学);Google(谷歌公司);Google DeepMind(谷歌DeepMind);University of Oxford(牛津大学);EPFL(洛桑联邦理工学院);ETH Zürich(苏黎世联邦理工学院);INSAIT(INSAIT研究院);Samsung Research(三星研究院)
Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric
University of Chinese Academy of Sciences(中国科学院大学);Institute of Optics and Electronics, Chinese Academy of Sciences(中国科学院光电技术研究所);Anhui University(安徽大学);Beijing Institute of Technology(北京理工大学);Peng Cheng Laboratory(鹏城实验室);Peking University(北京大学)