使用可能相关的测试证明软件可靠性:一种保守贝叶斯方法的见解
Demonstrating Software Reliability using Possibly Correlated Tests: Insights from a Conservative Bayesian Approach
AI总结:
本文提出保守贝叶斯方法,在软件执行可能相关时评估可靠性,揭示独立性假设的影响,并指出运行测试的徒劳性与故障敏感性,应用于核电站和自动驾驶安全。
摘要:
本文提出了用于对软件可靠性进行保守声明的贝叶斯技术,特别是在证据表明软件的执行并非统计独立的情况下。我们形式化了关于执行独立性的“怀疑”的非正式概念,并将这些怀疑纳入可靠性评估中。我们开发了技术来揭示独立性假设在多大程度上会削弱评估中的保守性,并确定了这种影响不显著的条件。这些技术——保守贝叶斯推断(CBI)方法的新扩展——给出了每次执行软件失败概率的保守置信界限。通过在两个应用领域——核电站安全和自动驾驶车辆(AV)安全——的说明,我们的分析揭示了:1)评估者在将系统投入运行测试之前应具备的信心。否则,这种测试是徒劳的——有利的运行测试证据最终会降低人们对系统足够可靠的信心;2)独立性假设有时支持保守声明;3)在某些场景下,观察系统无故障运行比观察到一些故障更能降低人们对系统的信心;4)建立对系统的信心对故障非常敏感——每次额外的故障意味着需要显著更多的运行测试,以支持可靠性声明。
英文摘要:
This paper presents Bayesian techniques for conservative claims about software reliability, particularly when evidence suggests the software's executions are not statistically independent. We formalise informal notions of "doubting" that the executions are independent, and incorporate such doubts into reliability assessments. We develop techniques that reveal the extent to which independence assumptions can undermine conservatism in assessments, and identify conditions under which this impact is not significant. These techniques - novel extensions of conservative Bayesian inference (CBI) approaches - give conservative confidence bounds on the software's failure probability per execution. With illustrations in two application areas - nuclear power-plant safety and autonomous vehicle (AV) safety - our analyses reveals: 1) the confidence an assessor should possess before subjecting a system to operational testing. Otherwise, such testing is futile - favourable operational testing evidence will eventually decrease one's confidence in the system being sufficiently reliable; 2) the independence assumption supports conservative claims sometimes; 3) in some scenarios, observing a system operate without failure gives less confidence in the system than if some failures had been observed; 4) building confidence in a system is very sensitive to failures - each additional failure means significantly more operational testing is required, in order to support a reliability claim.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。