arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.06999cs.CVcs.LG
收藏

通过损失检查识别目标检测数据集中的标签错误

Identifying Label Errors in Object Detection Datasets by Loss Inspection

University of Wuppertal(伍珀塔尔大学);ControlExpert GmbH(ControlExpert有限公司);University of Zagreb(萨格勒布大学)

Marius Schubert, Tobias Riedlinger, Karsten Kahl, Daniel Kröll, Sebastian Schoenen, Siniša Šegvić, Matthias Rottmann

AI总结:

该研究首次构建目标检测标签错误检测基准,提出基于两阶段检测器分类与回归损失之和的检测方法,在四类模拟错误和真实数据集上均优于基线,精确率最高达97%。

摘要:

为监督式目标检测标注数据集是一项枯燥且耗时的任务。标注过程中很容易引入错误,且这些错误在审核阶段常被忽略,会导致基准测试结果不准确,以及在含噪声标签上训练的深度神经网络性能下降。在本研究中,我们首次构建了一个面向目标检测数据集标签错误检测方法的基准(benchmark),同时提出了一种标签错误检测方法及若干基线方法。我们在标注质量良好的目标检测数据集的训练集和测试集上,模拟了四种不同类型的随机引入标签错误。对于我们提出的标签错误检测方法,我们假设给定一个两阶段目标检测器,并考虑两个阶段的分类损失与回归损失之和。这些损失是基于模型预测结果和包含模拟标签错误的噪声标签计算得出的,目的是检测出其中的错误标签。我们将该方法与三种基线方法进行了对比:一种不涉及深度学习的朴素方法、目标检测器的输出得分、以及分类softmax分布的熵。我们的方法性能优于所有基线,并且证明在所有被考量的方法中,我们的方法是唯一能有效检测全部四种类型标签错误的方法。此外,我们还在两类场景中检测真实标签错误:a)目标检测领域常用的测试数据集,b)一个专有数据集。在两种场景下我们都实现了较低的假阳性率,即检测标签错误的精确率在场景a中最高达71.5%,在场景b中为97%。

英文摘要:

Labeling datasets for supervised object detection is a dull and time-consuming task. Errors can be easily introduced during annotation and overlooked during review, yielding inaccurate benchmarks and performance degradation of deep neural networks trained on noisy labels. In this work, we for the first time introduce a benchmark for label error detection methods on object detection datasets as well as a label error detection method and a number of baselines. We simulate four different types of randomly introduced label errors on train and test sets of well-labeled object detection datasets. For our label error detection method we assume a two-stage object detector to be given and consider the sum of both stages' classification and regression losses. The losses are computed with respect to the predictions and the noisy labels including simulated label errors, aiming at detecting the latter. We compare our method to three baselines: a naive one without deep learning, the object detector's score and the entropy of the classification softmax distribution. We outperform all baselines and demonstrate that among the considered methods, ours is the only one that detects label errors of all four types efficiently. Furthermore, we detect real label errors a) on commonly used test datasets in object detection and b) on a proprietary dataset. In both cases we achieve low false positives rates, i.e., we detect label errors with a precision for a) of up to 71.5% and for b) with 97%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑