arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1608.01745cs.CV
收藏

玩与学:使用视频游戏训练计算机视觉模型

Play and Learn: Using Video Games to Train Computer Vision Models

University of British Columbia(不列颠哥伦比亚大学)

Alireza Shafaei, James J. Little, Mark Schmidt

摘要:

视频游戏是注释数据的一个引人注目的来源,因为它们可以很容易地为不同的任务提供细粒度的地面实况。然而,目前还不清楚合成生成的数据是否与真实世界的图像有足够的相似性,以提高计算机视觉模型在实践中的性能。我们目前的实验评估从视频游戏中提取的合成RGB图像上训练的系统对真实世界数据的有效性。我们从现代视频游戏中收集了超过60000个合成样本,这些样本的条件与现实世界的CamVid和Cityscapes数据集相似。我们提供了几个实验来证明合成生成的RGB图像可以用于提高深度神经网络在图像分割和深度估计方面的性能。这些结果表明,对于密集图像分类,在合成数据上训练的卷积网络实现了与在真实世界数据上训练的网络相似的测试误差。此外,如果应用简单的领域自适应技术,则合成生成的RGB图像可以提供与真实世界数据集相比类似或更好的结果。我们的研究结果表明,与游戏开发人员合作,提供一个可访问的界面来收集数据,可能是计算机视觉未来工作的一个富有成效的方向。

英文摘要:

Video games are a compelling source of annotated data as they can readily provide fine-grained groundtruth for diverse tasks. However, it is not clear whether the synthetically generated data has enough resemblance to the real-world images to improve the performance of computer vision models in practice. We present experiments assessing the effectiveness on real-world data of systems trained on synthetic RGB images that are extracted from a video game. We collected over 60000 synthetic samples from a modern video game with similar conditions to the real-world CamVid and Cityscapes datasets. We provide several experiments to demonstrate that the synthetically generated RGB images can be used to improve the performance of deep neural networks on both image segmentation and depth estimation. These results show that a convolutional network trained on synthetic data achieves a similar test error to a network that is trained on real-world data for dense image classification. Furthermore, the synthetically generated RGB images can provide similar or better results compared to the real-world datasets if a simple domain adaptation technique is applied. Our results suggest that collaboration with game developers for an accessible interface to gather data is potentially a fruitful direction for future work in computer vision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑