arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.14611cs.CV
收藏

学习模拟复杂场景

Learning to simulate complex scenes

Zhejiang University(浙江大学);Australian National University(澳大利亚国立大学)

Zhenfeng Xue, Weijie Mao, Liang Zheng

摘要:

像Unity这样的数据模拟引擎正在成为一个越来越重要的数据源,使我们能够方便地获取地面真实标签。此外,我们可以灵活地编辑引擎中图像的内容,例如对象(位置,方向)和环境(照明,遮挡)。当使用模拟数据作为训练集时,可以利用其可编辑内容来模拟真实世界数据的分布,从而减少合成域和真实域之间的内容差异。本文探讨了内容自适应的背景下,语义分割,其中复杂的街道场景是完全合成使用19类虚拟对象从第一人称驾驶员的角度和控制的23个属性。为了优化属性值并获得与真实世界数据相似的训练集,我们提出了一种可扩展的离散化和松弛(SDR)方法。在强化学习框架下,我们使用神经网络将属性优化表示为随机到优化的映射问题。我们的方法有三个特点。1)而不是编辑单个对象的属性,我们专注于全局属性,有很大的影响场景结构,如对象密度和照明。2)将属性量化为离散值,以减少搜索空间和训练复杂度。3)相关属性在一个组中联合优化,以避免无意义的场景结构,并找到更好的收敛点。实验表明,我们的系统可以产生合理的和有用的场景,从我们获得了良好的现实世界的分割精度相比,现有的合成训练集。

英文摘要:

Data simulation engines like Unity are becoming an increasingly important data source that allows us to acquire ground truth labels conveniently. Moreover, we can flexibly edit the content of an image in the engine, such as objects (position, orientation) and environments (illumination, occlusion). When using simulated data as training sets, its editable content can be leveraged to mimic the distribution of real-world data, and thus reduce the content difference between the synthetic and real domains. This paper explores content adaptation in the context of semantic segmentation, where the complex street scenes are fully synthesized using 19 classes of virtual objects from a first person driver perspective and controlled by 23 attributes. To optimize the attribute values and obtain a training set of similar content to real-world data, we propose a scalable discretization-and-relaxation (SDR) approach. Under a reinforcement learning framework, we formulate attribute optimization as a random-to-optimized mapping problem using a neural network. Our method has three characteristics. 1) Instead of editing attributes of individual objects, we focus on global attributes that have large influence on the scene structure, such as object density and illumination. 2) Attributes are quantized to discrete values, so as to reduce search space and training complexity. 3) Correlated attributes are jointly optimized in a group, so as to avoid meaningless scene structures and find better convergence points. Experiment shows our system can generate reasonable and useful scenes, from which we obtain promising real-world segmentation accuracy compared with existing synthetic training sets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑