arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.03102cs.CVcs.AI
收藏

基于人工判断的模糊图像鲁棒视觉事件分类

Ambiguous Images With Human Judgments for Robust Visual Event Classification

Johns Hopkins University(约翰斯·霍普金斯大学);Human Language Technology Center of Excellence(人类语言技术卓越中心)

Kate Sanders, Reno Kriz, Anqi Liu, Benjamin Van Durme

摘要:

当代视觉基准主要考虑人类可以实现近乎完美性能的任务。然而,人类经常会看到他们无法100%确定地分类的视觉数据,并且在标准视觉基准上训练的模型在评估这些数据时表现不佳。为了解决这个问题,我们介绍了一个程序,用于创建模糊图像的数据集,并使用它来产生SQUID-E(“Squidy”),从视频中提取的噪声图像的集合。所有图像都用地面真值进行注释,测试集用人类的不确定性判断进行注释。我们使用这个数据集来描述人类在视觉任务中的不确定性,并评估现有的视觉事件分类模型。实验结果表明,现有的视觉模型不足以提供有意义的输出模糊的图像,这种性质的数据集可以用来评估和改进这种模型,通过模型训练和模型校准的直接评估。这些发现激发了大规模模糊数据集的创建,并进一步研究了噪声视觉数据。

英文摘要:

Contemporary vision benchmarks predominantly consider tasks on which humans can achieve near-perfect performance. However, humans are frequently presented with visual data that they cannot classify with 100% certainty, and models trained on standard vision benchmarks achieve low performance when evaluated on this data. To address this issue, we introduce a procedure for creating datasets of ambiguous images and use it to produce SQUID-E ("Squidy"), a collection of noisy images extracted from videos. All images are annotated with ground truth values and a test set is annotated with human uncertainty judgments. We use this dataset to characterize human uncertainty in vision tasks and evaluate existing visual event classification models. Experimental results suggest that existing vision models are not sufficiently equipped to provide meaningful outputs for ambiguous images and that datasets of this nature can be used to assess and improve such models through model training and direct evaluation of model calibration. These findings motivate large-scale ambiguous dataset creation and further research focusing on noisy visual data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑