COCO等基准在目标检测中起着至关重要的作用。然而,现有的基准是不够的规模变化,其协议是不够的公平比较。在本文中,我们介绍了通用尺度目标检测基准(USB)。USB通过将COCO与最近提出的Waymo Open Dataset和Manga 109-s数据集相结合,在对象尺度和图像域上有所变化。为了实现公平比较和包容性研究,我们提出了培训和评估协议。他们有多个训练时期和评估图像分辨率的部门,如体育运动中的重量等级,以及跨训练协议的兼容性,如通用串行总线的向后兼容性。具体来说,我们要求参与者不仅报告更高协议(更长的培训)的结果,而且报告更低协议(更短的培训)的结果。使用提出的基准和协议,我们进行了广泛的实验,使用15种方法,发现现有的COCO偏置方法的弱点。该代码可在https://github.com/shinya7y/UniverseNet上获得。
英文摘要:
Benchmarks, such as COCO, play a crucial role in object detection. However, existing benchmarks are insufficient in scale variation, and their protocols are inadequate for fair comparison. In this paper, we introduce the Universal-Scale object detection Benchmark (USB). USB has variations in object scales and image domains by incorporating COCO with the recently proposed Waymo Open Dataset and Manga109-s dataset. To enable fair comparison and inclusive research, we propose training and evaluation protocols. They have multiple divisions for training epochs and evaluation image resolutions, like weight classes in sports, and compatibility across training protocols, like the backward compatibility of the Universal Serial Bus. Specifically, we request participants to report results with not only higher protocols (longer training) but also lower protocols (shorter training). Using the proposed benchmark and protocols, we conducted extensive experiments using 15 methods and found weaknesses of existing COCO-biased methods. The code is available at https://github.com/shinya7y/UniverseNet .
USB: A Unified Semi-supervised Learning Benchmark for Classification
Microsoft Research Asia(微软亚洲研究院);Westlake University(西湖大学);Tokyo Institute of Technology(东京工业大学);Carnegie Mellon University(卡内基梅隆大学);Max-Planck-Institut für Informatik(马克斯·普朗克信息学研究所);Tsinghua University(清华大学);Microsoft STCA(微软搜索技术中心);Nanjing University(南京大学);Nara Institute of Science and Technology(奈良先端科学技术大学院大学);Peking University(北京大学)
LEVIRDet: A Million-Scale 159-Category Dataset and Foundation Model for Universal Remote Sensing Object Detection
Shen Yuan Honors College, Beihang University(北京航空航天大学沈元荣誉学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology(中国空间技术研究院钱学森空间技术实验室)
Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark
Institute of Advanced Research in Artificial Intelligence(人工智能高等研究院);Helmholtz-Zentrum Dresden-Rossendorf(亥姆霍兹德累斯顿罗森多夫研究中心);Helmholtz Institute Freiberg for Resource Technology(亥姆霍兹弗赖贝格资源技术研究所)
UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery
School of Automation, Northwestern Polytechnical University(自动化学院,西北工业大学);College of Missile Engineering, Rocket Force University of Engineering(导弹工程学院,火箭兵工程大学);School of Artificial Intelligence, Chongqing University of Posts and Telecommunications(人工智能学院,重庆邮电大学)
The ULS23 Challenge: a Baseline Model and Benchmark Dataset for 3D Universal Lesion Segmentation in Computed Tomography
Radboud University Medical Center(拉德堡德大学医学中心);Jeroen Bosch Hospital(杰伦博世医院);University Medical Center Groningen(格罗宁根大学医学中心);Fraunhofer Institute for Digital Medicine MEVIS(弗劳恩霍夫数字医学梅维斯研究所)
mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning
Shenyang Institute of Computing Technology, Chinese Academy of Sciences(中国科学院沈阳计算技术研究所);Beijing Wenge Technology Co., Ltd(北京文格科技有限公司);University of Chinese Academy of Sciences(中国科学院大学)
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
University of Waterloo(滑铁卢大学);Georgia Institute of Technology(佐治亚理工学院);Hong Kong University of Science and Technology(香港科技大学);Google DeepMind(谷歌DeepMind)