arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.10416stat.AP
收藏

安全性分析方法的测试和评价方法

Methodology for Testing and Evaluation of Safety Analytics Approaches

Antonio R. Paiva, Ashutosh Tewari

摘要:

近年来,数据驱动的安全分析方法的发展显著增加。鉴于这些进展,有必要以原则性的方式评估这些方法,以确定其优点和局限性。为此,我们提出了一种基于模拟环境的评估方法,可以对安全分析方法进行全面评估。虽然用历史实地数据评估这些方法无疑是重要的,但这种评估的统计功效有限,因为它只对应于一种实现。所提出的方法能够验证大量的实现,从而规避了历史数据评价的统计局限性。此外,通过使用模拟环境,能够清楚地区分观察到的数据的变异性和安全方法之间的性能差异。模拟环境通过在受控环境下比较各种方法来做到这一点,从而对潜在的长期效益进行公平和系统的评估。我们通过案例研究比较了几种候选安全分析方法,展示了所提出的方法的实用性。这些方法的不同之处在于它们如何吸收现场安全数据以评估安全风险并提出缓解措施。我们表明,所提出的方法确实揭示了有用的见解,并量化了不同方法的相对优点和缺点,否则很难在现实世界中客观地确定。

英文摘要:

There has been a significant increase in the development of data-driven safety analytics approaches in recent years. In light of these advances it has become imperative to evaluate such approaches in a principled way to determine their merits and limitations. To that end, we propose an evaluation methodology underpinned by a simulated environment that allows for a comprehensive assessment of safety analytics approaches. While assessing those approaches with historical field data is undoubtedly important, such an assessment has limited statistical power because it corresponds to only one realization. The proposed methodology enables validation over a large number of realizations, thereby circumventing the statistical limitations of evaluation on historical data. Moreover, by using a simulated environment one is able to clearly distinguish between the variability in the observed data and differences in performance between safety approaches. A simulated environment does this by comparing the approaches under controlled circumstances, resulting in a fair and systematic evaluation of the potential long-term benefits. We demonstrate the utility of the proposed methodology via a case study that compares a few candidate safety analytics approaches. These approaches differ in how they assimilate field safety data to assess safety risk and suggest mitigative actions. We show that the proposed methodology indeed reveals useful insights and quantifies the relative merits and drawbacks of the different approaches, which would be otherwise difficult to objectively determine in a real-world scenario.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑