arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.11730cs.LGcs.AIcs.CYcs.HCstat.ML
收藏

可解释规划策略的自动发现

Automatic Discovery of Interpretable Planning Strategies

Julian Skirzyński, Frederic Becker, Falk Lieder

摘要:

在做决策时,人们往往会忽视关键信息,或者被无关信息过度左右。减轻这些偏见的一种常见方法是为决策者,特别是医生等专业人员提供决策辅助工具,如决策树和流程图。设计有效的决策辅助工具是一个难题。我们建议,最近开发的强化学习方法,发现聪明的策略,良好的决策,可以部分利用,以协助人类专家在这个设计过程中。利用上述方法的最大障碍之一是,他们学到的政策对人们来说是不透明的。为了解决这个问题,我们引入了AI-Interpret:一种将特殊策略转换为简单且可解释的描述的通用方法。我们的算法结合了模仿学习和程序归纳的最新进展,一个新的聚类方法,用于识别一个大的子集的演示,可以准确地描述了一个简单的,高性能的决策规则。我们评估我们的新算法,并采用它来翻译通过元级强化学习发现的信息获取策略。大型行为实验的结果表明,将AI-Interpret生成的决策规则作为流程图提供,可以显着改善人们在三种不同类别的顺序决策问题中的规划策略和决策。此外,另一项实验表明,这种方法比通过给人们提供绩效反馈来培训他们更有效。最后,一系列的消融研究证实了AI解释对于发现可解释的决策规则至关重要。我们的结论是,本文提出的方法和研究结果是利用自动策略发现来改善人类决策的重要一步。

英文摘要:

When making decisions, people often overlook critical information or are overly swayed by irrelevant information. A common approach to mitigate these biases is to provide decision-makers, especially professionals such as medical doctors, with decision aids, such as decision trees and flowcharts. Designing effective decision aids is a difficult problem. We propose that recently developed reinforcement learning methods for discovering clever heuristics for good decision-making can be partially leveraged to assist human experts in this design process. One of the biggest remaining obstacles to leveraging the aforementioned methods is that the policies they learn are opaque to people. To solve this problem, we introduce AI-Interpret: a general method for transforming idiosyncratic policies into simple and interpretable descriptions. Our algorithm combines recent advances in imitation learning and program induction with a new clustering method for identifying a large subset of demonstrations that can be accurately described by a simple, high-performing decision rule. We evaluate our new algorithm and employ it to translate information-acquisition policies discovered through metalevel reinforcement learning. The results of large behavioral experiments showed that prividing the decision rules generated by AI-Interpret as flowcharts significantly improved people's planning strategies and decisions across three diferent classes of sequential decision problems. Moreover, another experiment revealed that this approach is significantly more effective than training people by giving them performance feedback. Finally, a series of ablation studies confirmed that AI-Interpret is critical to the discovery of interpretable decision rules. We conclude that the methods and findings presented herein are an important step towards leveraging automatic strategy discovery to improve human decision-making.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑