ColorSense：机器视觉识别中的色觉研究
ColorSense: A Study on Color Vision in Machine Visual Recognition
AI总结：
为探究色觉对机器视觉识别的影响，构建多用途数据集ColorSense（含11万条人类注释），通过颜色辨别水平分析其对模型性能的影响，发现色觉偏差影响显著且现有缓解技术效果有限。
摘要：
色觉对人类视觉感知至关重要，但其对机器感知的影响仍未得到充分探索。对于辅助驾驶和手术等安全关键任务，理解色觉在机器感知中的作用的需求日益迫切，但缺乏合适的数据集。为填补这一空白，我们通过从流行的视觉识别基准中收集110,000个非平凡的前景和背景颜色标签的人类注释，构建了多用途数据集ColorSense。为研究色觉对机器感知的影响，我们根据图像的主要前景和背景颜色为每个图像分配一个颜色辨别水平，并利用它来研究色觉对机器感知的影响。我们通过证明颜色辨别水平对主流机器感知模型的性能具有主导影响，验证了我们数据集的使用。具体而言，我们通过考虑模型架构、训练目标、模型大小、训练数据和任务复杂性等关键因素，检验了机器视觉的感知能力。此外，为研究颜色和环境因素如何影响机器感知中视觉识别的鲁棒性，我们将ColorSense数据集与图像损坏相结合，进行了更全面的视觉感知评估。我们的研究结果表明，分类和定位等目标识别任务易受色觉偏差的影响，尤其是在车辆类别等高风险案例中，而数据增强等先进缓解技术仅带来边际改进。我们的分析强调了在现实应用中对机器感知模型性能评估的新方法的需求。最后，我们提出了ColorSense的各种潜在应用，例如研究虚假相关性。
英文摘要：
Color vision is essential for human visual perception, but its impact on machine perception is still underexplored. There has been an intensified demand for understanding its role in machine perception for safety-critical tasks such as assistive driving and surgery but lacking suitable datasets. To fill this gap, we curate multipurpose datasets ColorSense, by collecting 110,000 non-trivial human annotations of foreground and background color labels from popular visual recognition benchmarks. To investigate the impact of color vision on machine perception, we assign each image a color discrimination level based on its dominant foreground and background colors and use it to study the impact of color vision on machine perception. We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of mainstream machine perception models. Specifically, we examine the perception ability of machine vision by considering key factors such as model architecture, training objective, model size, training data, and task complexity. Furthermore, to investigate how color and environmental factors affect the robustness of visual recognition in machine perception, we integrate our ColorSense datasets with image corruptions and perform a more comprehensive visual perception evaluation. Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias, especially for high-stakes cases such as vehicle classes, and advanced mitigation techniques such as data augmentation and so on only give marginal improvement. Our analyses highlight the need for new approaches toward the performance evaluation of machine perception models in real-world applications. Lastly, we present various potential applications of ColorSense such as studying spurious correlations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
