自动驾驶测试中用于缓解神经现实差距输入的质量指标评估
Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing
AI总结：
研究I2I神经转换质量对自动驾驶测试中sim2real差距缓解的影响，发现现有评估指标与ADS行为不一致，提出结合语义元素的特定任务微调感知指标以可靠评估I2I技术。
摘要：
自动驾驶系统（ADS）的仿真测试是行业标准，作为真实世界测试的受控、安全且具有成本效益的替代方案。尽管有这些优势，虚拟仿真往往无法准确复现真实世界条件，如图像保真度、纹理表示和环境准确性。这可能导致ADS在仿真域和真实世界域之间的行为产生显著差异，这种现象被称为sim2real差距。研究人员已使用图像到图像（I2I）神经转换来缓解sim2real差距，通过将合成数据转换为更真实的现实世界条件表示，从而增强仿真环境的真实性。然而，尽管这些技术前景广阔，但可能会在生成数据中引入伪影、失真或不一致性，进而影响ADS测试的有效性。在我们的实证研究中，我们利用文献中一系列既定指标，调查了图像到图像（I2I）技术的质量如何影响sim2real差距的缓解。我们在模型级别评估了两种流行的生成式I2I架构pix2pix和CycleGAN，涉及两项ADS感知任务，即车辆检测和端到端车道保持，使用了配对的仿真和真实世界数据集。我们的研究结果表明，I2I架构的有效性在不同的ADS任务中存在差异，且现有的评估指标与ADS行为并不一致。因此，我们对感知指标进行了特定任务的微调，获得了更强的相关性。我们的研究结果表明，结合语义元素并为每个任务定制的感知指标，有助于选择最合适的I2I技术，从而可靠地评估sim2real差距的缓解效果。
英文摘要：
Simulation-based testing of automated driving systems (ADS) is the industry standard, being a controlled, safe, and cost-effective alternative to real-world testing. Despite these advantages, virtual simulations often fail to accurately replicate real-world conditions like image fidelity, texture representation, and environmental accuracy. This can lead to significant differences in ADS behavior between simulated and real-world domains, a phenomenon known as the sim2real gap. Researchers have used Image-to-Image (I2I) neural translation to mitigate the sim2real gap, enhancing the realism of simulated environments by transforming synthetic data into more authentic representations of real-world conditions. However, while promising, these techniques may potentially introduce artifacts, distortions, or inconsistencies in the generated data that can affect the effectiveness of ADS testing. In our empirical study, we investigated how the quality of image-to-image (I2I) techniques influences the mitigation of the sim2real gap, using a set of established metrics from the literature. We evaluated two popular generative I2I architectures, pix2pix, and CycleGAN, across two ADS perception tasks at a model level, namely vehicle detection and end-to-end lane keeping, using paired simulated and real-world datasets. Our findings reveal that the effectiveness of I2I architectures varies across different ADS tasks, and existing evaluation metrics do not consistently align with the ADS behavior. Thus, we conducted task-specific fine-tuning of perception metrics, which yielded a stronger correlation. Our findings indicate that a perception metric that incorporates semantic elements, tailored to each task, can facilitate selecting the most appropriate I2I technique for a reliable assessment of the sim2real gap mitigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
