通过动态安全包线监督不安全系统
Oversight of Unsafe Systems via Dynamic Safety Envelopes
摘要:
本文回顾了人在回路对于防止机器学习的广泛理解的故障模式是至关重要的,而不是一个实际的解决方案的原因。在此之后,我们回顾了两个当前的探试法来解决这个问题。第一个是可证明的安全包络,这是可能的,只有当系统的动态是完全已知的,但可以是有用的安全保证时,最佳的行为是基于机器学习的理解不充分的安全特性。第二种是更简单的断路器模型,它可以通过停止系统来阻止或防止灾难性的结果,而无需任何特定的系统模型。本文提出使用启发式的动态安全信封,这是这些方法之间的一个合理的中间点,允许人类监督,而不会遇到一些更困难的问题,人类在环系统。最后,本文的结论是如何使用这种方法来治理系统,否则不安全的系统部署。
英文摘要:
This paper reviews the reasons that Human-in-the-Loop is both critical for preventing widely-understood failure modes for machine learning, and not a practical solution. Following this, we review two current heuristic methods for addressing this. The first is provable safety envelopes, which are possible only when the dynamics of the system are fully known, but can be useful safety guarantees when optimal behavior is based on machine learning with poorly-understood safety characteristics. The second is the simpler circuit breaker model, which can forestall or prevent catastrophic outcomes by stopping the system, without any specific model of the system. This paper proposes using heuristic, dynamic safety envelopes, which are a plausible halfway point between these approaches that allows human oversight without some of the more difficult problems faced by Human-in-the-Loop systems. Finally, the paper concludes with how this approach can be used for governance of systems where otherwise unsafe systems are deployed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。