Recently, there has been a significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is deep learning testing, where adversarial examples (a.k.a.~bugs) of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the commonly used neuron coverage metrics by existing DL testing approaches are not correlated to model robustness. It is also not an effective measurement on the confidence of the model robustness after testing. In this work, we address this gap by proposing a novel testing framework called Robustness-Oriented Testing (RobOT). A key part of RobOT is a quantitative measurement on 1) the value of each test case in improving model robustness (often via retraining), and 2) the convergence quality of the model robustness improvement. RobOT utilizes the proposed metric to automatically generate test cases valuable for improving model robustness. The proposed metric is also a strong indicator on how well robustness improvement has converged through testing. Experiments on multiple benchmark datasets confirm the effectiveness and efficiency of RobOT in improving DL model robustness, with 67.02% increase on the adversarial robustness that is 50.65% higher than the state-of-the-art work DeepGini.
Improving the Testability of Object-oriented Software during Testing and Debugging Processes
可测试性是在程序中存在错误的情况下，测试是否会检测到错误的概率。如何有效地发现故障取决于软件的可测试性。许多研究人员提出了定性和定量的技术来提高和衡量软件的可测试性。在文献中，大量的可靠性增长模型已被用于评估和衡量软件在测试和运行阶段的定量质量评估。了解软件故障分布及其复杂性可以提高软件的可测试性。通过了解故障分布和故障复杂度，可以使测试工作量的分配变得容易，这将简化从软件中发现故障的过程。因此，软件的可测试性将得到改善。文中给出了该模型的参数以及不同复杂度的故障在软件中所占的比例，并利用在开源环境下开发的两个面向对象软件MySQL for python和Squirrel SQL Client的故障数据进行了估计
Applications of different metaheuristic techniques for finding optimal tst order during integration testing of object oriented systems and their comparative study
Before Autonomy Takes Control: Software Testing in Robotics
Ruhr University Bochum Bochum Germany；University of Bremen Bremen Germany；IT University of Copenhagen Copenhagen Denmark；Chalmers\,|\,Gothenburg University Gothenburg Sweden；Ruhr University Bochum；University of Bremen；IT University of Copenhagen；Chalmers\,|\,Gothenburg University
Vision Language Model-based Testing of Industrial Autonomous Mobile Robots
Simula Research Laboratory and University of Oslo（Simula研究实验室和奥斯陆大学）；Mondragon University（蒙dragon大学）；Simula Research Laboratory（Simula研究实验室）；PAL Robotics（PAL机器人技术）
VISOR: A Vision-Language Model-based Test Oracle for Testing Robots
Simula Research Laboratory（Simula研究实验室）；Oslo Metropolitan University（奥斯陆理工大学）；Mondragon University（蒙dragon大学）；National Institute of Informatics（国立信息研究所）
