在不确定环境中实现自主决策的原则性问责,需要区分有意造成的结果、因设计疏忽导致的结果和真正的事故。我们提出通过一种对意图行为证据的量化度量来分析自主智能体的行为。我们将不确定环境建模为马尔可夫决策过程(MDP)。对于给定场景,我们依靠概率模型检验来计算智能体影响到达某一特定事件的能力,并将其称为能动范围(scope of agency)。我们认为,如果能动范围较高,且智能体的决策接近到达该事件的最优决策,则存在意图行为的证据。我们的方法应用反事实推理来自动生成相关场景,通过分析这些场景来提高评估的可信度。在一个案例研究中,我们展示了该方法如何区分“有意”和“意外”的交通碰撞。
英文摘要:
Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. We model an uncertain environment as a Markov Decision Process (MDP). For a given scenario, we rely on probabilistic model checking to compute the ability of the agent to influence reaching a certain event. We call this the scope of agency. We say that there is evidence of intentional behavior if the scope of agency is high and the decisions of the agent are close to being optimal for reaching the event. Our method applies counterfactual reasoning to automatically generate relevant scenarios that can be analyzed to increase the confidence of our assessment. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions.
MSC-Bench: Benchmarking and Analyzing Multi-Sensor Corruption for Driving Perception
Beijing Academy of Artificial Intelligence(北京人工智能研究院);IQIYI(爱奇艺);Institute of Automation CAS(中国科学院自动化研究所);Samsung(三星);The University of Sydney(悉尼大学);National University of Singapore(新加坡国立大学);Institute of Information Engineering CAS(中国科学院信息工程研究所);Hefei University of Technology(合肥工业大学)