发表机构
ITPower Solutions GmbH（ITPower Solutions 有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对ML系统测试失效问题，提出概率扩展本体（PEONs）对运行设计域进行概率建模，实现代表性测试采样、严格结束标准及数据平衡评估，并以自动列车运行验证。
AI 中文摘要
传统的测试流程在应用于基于机器学习的系统（如车辆中的障碍物检测）时很快就会失效：如果在测试中未检测到障碍物，经典的错误修复是不可能的，AI系统将始终存在缺陷。因此，测试结果只能进行统计解释，这反过来要求测试集不仅相对于系统的运行设计域（ODD）是完整的，而且对其具有代表性。为此，我们引入了概率扩展本体（PEONs）：描述ODD的本体，并对其所诱导的划分附加概率分布。无需维护不可行的条件概率表，只需指定边际分布和功能描述的依赖关系；基于耦合和最优传输的算法将此规范完善为贝叶斯网络。从PEON中，我们推导出代表性测试用例的采样、针对给定质量目标和显著性水平的严格测试结束标准，以及重新评估现有测试结果和评估训练数据平衡性的方法。我们以自动列车运行为例，展示了复杂ODD的实际建模。
英文摘要
The conventional testing process quickly fails when applied to ML-based systems such as obstacle detection in vehicles: if an obstacle is not detected in a test, classical bug fixing is impossible and an AI system will always retain shortcomings. Test results can therefore only be interpreted statistically, which in turn requires test sets that are not only complete with respect to the operational design domain (ODD) of the system, but also representative of it. To this end, we introduce probabilistically extended ontologies (PEONs): ontologies describing the ODD, augmented with a probability distribution over the partitioning they induce. Instead of unmaintainable conditional probability tables, only marginal distributions and functionally described dependencies need to be specified; algorithms based on couplings and optimal transport complete this specification to a Bayesian network. From a PEON we derive the sampling of representative test cases, rigorous end-of-test criteria for given quality targets and significance levels, and methods for re-evaluating existing test results and for assessing the balance of training data. We demonstrate the practical modelling of a complex ODD using the example of automatic train operation.
Comments50 pages, 43 figures, Technical Report
