AI 中文总结
本文针对ImageNet-1k数据集的标签噪声问题，构建ReImageNet数据集，采用人机协作反复优化的标注流程，使模型准确率显著提升，相关资源已公开。
AI 中文摘要
ImageNet-1k的Top-1准确率仍然是视觉识别领域最常报告的指标。该数据集的质量问题已被多次报道，但2012年原始的带噪声标签仍被广泛使用。本文开展了远超以往修正尝试的全面工作，旨在获取准确且完整的ImageNet-1k验证集标注。成果ReImageNet包含多标签修正、目标定位、修订后的类别定义以及语义属性（文本识别、呈现方式、反射、人群、主导对象）。重新标注结果显示，约12%的原始ImageNet-1k标签错误，33.3%的图像为多标签，3.8%的图像不包含ImageNet-1k类别的任何对象。使用新标签后，监督模型的Top-1准确率提升最高达1.2%，多模态大语言模型（MLLMs）的准确率提升5-6%。本文认为，ImageNet规模的标注工作无法通过一次完成，因为错误和定义性问题仅能在标注过程中被发现，因此本文的标注流程围绕反复优化与错误检查构建。研究发现，人类与大语言模型（LLM）在合适工具支持下的协作，代表了当前该规模标注的质量上限。ImageNet-1k的问题会传播到其衍生测试集，表明该问题是结构性的，而非特定于某个基准。所有标注、类别定义、标注指南及分析代码均已公开。项目页面：this https URL；标注：this https URL；代码：this https URL
英文摘要
Top-1 accuracy on ImageNet-1k remains the most commonly reported metric in visual recognition. Quality issues with the dataset have been repeatedly reported, yet the original 2012 noisy labels are still predominantly used. The paper presents a comprehensive effort, which goes well beyond prior correction attempts, towards obtaining accurate and complete ImageNet-1k validation set annotations. The result, ReImageNet, includes multilabel correction, object localization, revised class definitions, and semantic attributes (text-recognition, rendition, reflection, crowd, dominant). The reannotation reveals that approximately 12% of the original ImageNet-1k labels are incorrect, 33.3% of images are multilabel and 3.8% contain no object from an ImageNet-1k class. With the new labels, top-1 accuracy increases by up to 1.2% for supervised models and by 5-6% for MLLMs. We argue that annotation at ImageNet scale cannot realistically be completed in one pass, as errors and definitional issues are discovered only through annotating, and we build our pipeline around repeated refinement and error checking. We observed that human and LLM collaboration with appropriate tooling represents the current quality ceiling for annotation at this scale. ImageNet-1k issues propagate into its derivative test sets, indicating that the problem is structural rather than specific to any single benchmark. All annotations, class definitions, guidelines, and analysis code have been publicly released. Project page: https://vrg.fel.cvut.cz/reimagenet Annotations: https://huggingface.co/datasets/vrg-prague/ReImageNet Code: https://github.com/klarajanouskova/ImageNet
Comments25 pages, 16 figures, 8 tables. Project page: https://vrg.fel.cvut.cz/reimagenet
