arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.02562cs.LGcs.CVcs.NEstat.ML
收藏

TFCheck:一个用于检测神经网络程序训练问题的TensorFlow库

TFCheck : A TensorFlow Library for Detecting Training Issues in Neural Network Programs

Polytechnique Montréal(蒙特利尔高等理工学院)

Houssem Ben Braiek, Foutse Khomh

摘要:

越来越多的机器学习(ML)模型被纳入自动驾驶汽车等安全关键系统中,导致了基于多模型的ML测试技术的发展。这些测试技术的一个共同点是他们假设培训计划是足够的,没有错误。这些技术仅关注于使用手动标记的数据或自动生成的数据来评估构建的模型的性能。然而,他们对培训计划的假设并不总是正确的,因为培训计划可能包含不一致和错误。在本文中,我们研究了ML程序中的训练问题,并提出了一个可用于自动检测识别问题的验证例程目录。我们在一个名为TFCheck的基于Tensorflow的库中实现了例程。使用TFCheck,从业者可以自动检测上述问题。为了评估TFCheck的有效性,我们对真实世界、突变体和合成训练程序进行了案例研究。结果表明,TFCheck可以成功地检测ML代码实现中的训练问题。

英文摘要:

The increasing inclusion of Machine Learning (ML) models in safety critical systems like autonomous cars have led to the development of multiple model-based ML testing techniques. One common denominator of these testing techniques is their assumption that training programs are adequate and bug-free. These techniques only focus on assessing the performance of the constructed model using manually labeled data or automatically generated data. However, their assumptions about the training program are not always true as training programs can contain inconsistencies and bugs. In this paper, we examine training issues in ML programs and propose a catalog of verification routines that can be used to detect the identified issues, automatically. We implemented the routines in a Tensorflow-based library named TFCheck. Using TFCheck, practitioners can detect the aforementioned issues automatically. To assess the effectiveness of TFCheck, we conducted a case study with real-world, mutants, and synthetic training programs. Results show that TFCheck can successfully detect training issues in ML code implementations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑