自动驾驶汽车软件中基于成本效益的仿真测试选择
Cost-effective Simulation-based Test Selection in Self-driving Cars Software
摘要:
仿真环境对于自动驾驶汽车(SDC)等复杂网络物理系统的持续开发至关重要。SDC基于仿真的测试的先前结果已经表明,许多自动生成的测试并不强烈地有助于识别SDC故障,因此并不有助于提高SDC的质量。因为运行这样的“无信息”测试通常会导致计算资源的浪费和SDC测试成本的急剧增加,测试人员应该避免它们。然而,在运行测试之前识别“无信息”测试仍然是一个公开的挑战。因此,本文提出了SDCScissor,一个框架,利用机器学习(ML),以确定SDC测试,不太可能检测到故障的SDC软件测试,从而使测试人员跳过他们的执行,并大大提高成本效益的SDC软件的基于模拟的测试。我们在两个大型数据集上使用六个ML模型进行了22'652次测试,结果显示SDC-Scissor的分类F1得分高达96%。此外,我们的研究结果表明,SDC-Scissor在识别每个时间单位更多的失败测试方面优于随机基线。网页和视频:https://github.com/ChristianBirchler/sdc-scissor
英文摘要:
Simulation environments are essential for the continuous development of complex cyber-physical systems such as self-driving cars (SDCs). Previous results on simulation-based testing for SDCs have shown that many automatically generated tests do not strongly contribute to identification of SDC faults, hence do not contribute towards increasing the quality of SDCs. Because running such "uninformative" tests generally leads to a waste of computational resources and a drastic increase in the testing cost of SDCs, testers should avoid them. However, identifying "uninformative" tests before running them remains an open challenge. Hence, this paper proposes SDCScissor, a framework that leverages Machine Learning (ML) to identify SDC tests that are unlikely to detect faults in the SDC software under test, thus enabling testers to skip their execution and drastically increase the cost-effectiveness of simulation-based testing of SDCs software. Our evaluation concerning the usage of six ML models on two large datasets characterized by 22'652 tests showed that SDC-Scissor achieved a classification F1-score up to 96%. Moreover, our results show that SDC-Scissor outperformed a randomized baseline in identifying more failing tests per time unit. Webpage & Video: https://github.com/ChristianBirchler/sdc-scissor
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。