像标注者一样思考：数据集标注指令的生成
Thinking Like an Annotator: Generation of Dataset Labeling Instructions
AI总结：
本文提出标注指令生成新任务及无需训练的框架，利用预训练视觉语言模型和快速检索系统，为数据集类别生成多样化视觉与文本表示，在NuImages和COCO上分别超越最强基线7.06和12.9 mAP。
摘要：
大规模数据集对于现代深度学习至关重要。倡导者认为，理解这些方法需要数据集透明度（例如“数据集的策划、动机、组成、收集过程等”）。然而，几乎没有人建议发布提供给标注者的详细定义和视觉类别示例——这些信息对于理解每个数据集中现有标注的结构至关重要。这些标签是公开数据集的核心，但很少有数据集包含用于生成它们的指令。我们引入了一个新任务，即标注指令生成，以解决公开标注指令缺失的问题。在标注指令生成中，我们获取一个标注合理的数据集并：1）生成一组在视觉上代表数据集中每个类别的示例；2）提供与每个示例相对应的文本标签。我们引入了一个无需模型训练即可解决此任务的框架，并包含一个新创建的快速检索系统，该系统利用了大型预训练视觉和语言模型。该框架充当人类标注者的代理，既可以帮助生成最终的标注指令集，又可以评估其质量。我们的框架生成了数据集类别的多种多样化视觉和文本表示。优化后的指令集在5折交叉验证中，对于NuImages比我们最强的基线高出7.06 mAP，对于COCO高出12.9 mAP。
英文摘要：
Large-scale datasets are essential to modern day deep learning. Advocates argue that understanding these methods requires dataset transparency (e.g. "dataset curation, motivation, composition, collection process, etc..."). However, almost no one has suggested the release of the detailed definitions and visual category examples provided to annotators - information critical to understanding the structure of the annotations present in each dataset. These labels are at the heart of public datasets, yet few datasets include the instructions that were used to generate them. We introduce a new task, Labeling Instruction Generation, to address missing publicly available labeling instructions. In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples. We introduce a framework that requires no model training to solve this task and includes a newly created rapid retrieval system that leverages a large, pre-trained vision and language model. This framework acts as a proxy to human annotators that can help to both generate a final labeling instruction set and evaluate its quality. Our framework generates multiple diverse visual and text representations of dataset categories. The optimized instruction set outperforms our strongest baseline across 5 folds by 7.06 mAP for NuImages and 12.9 mAP for COCO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
