基于搜索的信息物理系统测试环境自动生成框架
A Search-Based Framework for Automatic Generation of Testing Environments for Cyber-Physical Systems
摘要:
许多现代网络物理系统都采用了计算机视觉技术、复杂的传感器和先进的控制软件,使它们能够自主地与环境进行交互。测试这样的系统提出了许多挑战:不仅要改变系统输入,而且要考虑周围环境。已经开发了一些工具来测试系统模型,以确定是否存在伪造其要求的可能输入。然而,它们并不直接适用于自主网络物理系统,因为其模型的输入是在虚拟环境中运行时生成的。在本文中,我们的目标是设计一个搜索为基础的框架,命名为AmbieGen,为自主网络物理系统产生不同的故障揭示测试场景。这些场景表示自主智能体在其中操作的环境。该框架应适用于生成不同类型的环境。为了生成测试场景,我们利用NSGA II算法有两个目标。第一个目标评估观察到的系统行为与其预期行为的偏差。第二个目标是测试用例的多样性,计算为与参考测试用例的Jaccard距离。我们评估AmbieGen的三个场景生成的案例研究,即智能恒温器,机器人避障系统,和车辆车道保持辅助系统。我们比较了三种配置的AmbieGen:基于单目标遗传算法,多目标,随机搜索。单目标和多目标配置都优于随机搜索。多目标配置可以找到与单个目标相同质量的个体,在相同的时间预算内产生更多独特的测试场景。
英文摘要:
Many modern cyber physical systems incorporate computer vision technologies, complex sensors and advanced control software, allowing them to interact with the environment autonomously. Testing such systems poses numerous challenges: not only should the system inputs be varied, but also the surrounding environment should be accounted for. A number of tools have been developed to test the system model for the possible inputs falsifying its requirements. However, they are not directly applicable to autonomous cyber physical systems, as the inputs to their models are generated while operating in a virtual environment. In this paper, we aim to design a search based framework, named AmbieGen, for generating diverse fault revealing test scenarios for autonomous cyber physical systems. The scenarios represent an environment in which an autonomous agent operates. The framework should be applicable to generating different types of environments. To generate the test scenarios, we leverage the NSGA II algorithm with two objectives. The first objective evaluates the deviation of the observed system behaviour from its expected behaviour. The second objective is the test case diversity, calculated as a Jaccard distance with a reference test case. We evaluate AmbieGen on three scenario generation case studies, namely a smart-thermostat, a robot obstacle avoidance system, and a vehicle lane keeping assist system. We compared three configurations of AmbieGen: based on a single objective genetic algorithm, multi objective, and random search. Both single and multi objective configurations outperform the random search. Multi objective configuration can find the individuals of the same quality as the single objective, producing more unique test scenarios in the same time budget.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。