arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.07083cs.LGcs.SE
收藏

一种用于提升 DNN 性能的基于场景的功能测试方法

A Scenario-Based Functional Testing Approach to Improving DNN Performance

School of Engineering, Computing and Mathematics, Oxford Brookes University(牛津布鲁克斯大学工程、计算与数学学院)

Hong Zhu, Thi Minh Tam Tran, Aduen Benjumea, Andrew Bradley

AI总结:

本文提出迭代式场景功能测试方法,通过识别、确认薄弱场景并结合迁移学习进行针对性再训练,在自动驾驶赛车 DNN 感知系统上验证了性能提升且资源消耗更低。

摘要:

本文提出一种基于场景的功能测试方法,用于增强机器学习(ML)应用的性能。所提出的方法是一个迭代过程:首先在各种场景下测试 ML 模型,以识别薄弱环节;随后对疑似薄弱场景进行进一步测试,并从统计上评估模型在这些场景上的性能,以确认诊断。一旦测试结果确认了薄弱场景的诊断,就对模型进行治疗:采用迁移学习技术,以原始模型为基础重新训练模型,并应用一组专门针对被治疗场景的训练数据,加上从原始训练数据集中随机选择的一个训练数据子集,以防止所谓的灾难性遗忘效应。最后,在治疗之后,通过在被治疗场景以及其他场景上进行测试,再次评估模型,以检查治疗是否有效且未造成副作用。本文报告了一个使用真实 ML 深度神经网络(DNN)模型的案例研究,该模型是自动驾驶赛车的感知系统。研究表明,该方法能够有效提升 DNN 模型的性能。它提供了一种增强 ML 模型性能的高效方法,与从头开始重新训练相比,所需的人力和计算资源少得多。

英文摘要:

This paper proposes a scenario-based functional testing approach for enhancing the performance of machine learning (ML) applications. The proposed method is an iterative process that starts with testing the ML model on various scenarios to identify areas of weakness. It follows by a further testing on the suspected weak scenarios and statistically evaluate the model's performance on the scenarios to confirm the diagnosis. Once the diagnosis of weak scenarios is confirmed by test results, the treatment of the model is performed by retraining the model using a transfer learning technique with the original model as the base and applying a set of training data specifically targeting the treated scenarios plus a subset of training data selected at random from the original train dataset to prevent the so-call catastrophic forgetting effect. Finally, after the treatment, the model is assessed and evaluated again by testing on the treated scenarios as well as other scenarios to check if the treatment is effective and no side effect caused. The paper reports a case study with a real ML deep neural network (DNN) model, which is the perception system of an autonomous racing car. It is demonstrated that the method is effective in the sense that DNN model's performance can be improved. It provides an efficient method of enhancing ML model's performance with much less human and compute resource than retrain from scratch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑