arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.10407cs.LGcs.AIcs.CY
收藏

从演示中学习行为软约束

Learning Behavioral Soft Constraints from Demonstrations

Tulane University(杜兰大学);University of Brescia(布雷西亚大学);University of West Florida(西佛罗里达大学);IHMC(人类与机器认知研究所);IBM Research(IBM研究院)

Arie Glazier, Andrea Loreggia, Nicholas Mattei, Taher Rahgooy, Francesca Rossi, Brent Venable

摘要:

许多现实生活中的场景要求人类做出艰难的权衡:我们是总是遵守所有的交通规则,还是在紧急情况下违反限速?这些情景迫使我们评估集体规则和规范与我们自己的个人目标和愿望之间的权衡。为了创建有效的人工智能-人类团队,我们必须为人工智能智能体配备一个模型,说明当存在隐式和显式规则和约束时,人类如何在复杂环境中进行权衡。配备这些模型的智能体将能够反映人类行为和/或将人类注意力吸引到可以改善决策的情况。为此,我们提出了一种新的逆强化学习(IRL)方法:最大熵逆软约束IRL(MESC-IRL),学习隐式的硬和软约束的状态,动作和状态特征的演示中确定性和非确定性的环境建模为马尔可夫决策过程(MDP)。我们的方法使代理隐式学习人类的约束和愿望,而不需要明确的建模代理设计师和环境之间转移这些约束。我们的新方法推广了以前的工作,只考虑确定性的硬约束,并实现了最先进的性能。

英文摘要:

Many real-life scenarios require humans to make difficult trade-offs: do we always follow all the traffic rules or do we violate the speed limit in an emergency? These scenarios force us to evaluate the trade-off between collective rules and norms with our own personal objectives and desires. To create effective AI-human teams, we must equip AI agents with a model of how humans make these trade-offs in complex environments when there are implicit and explicit rules and constraints. Agent equipped with these models will be able to mirror human behavior and/or to draw human attention to situations where decision making could be improved. To this end, we propose a novel inverse reinforcement learning (IRL) method: Max Entropy Inverse Soft Constraint IRL (MESC-IRL), for learning implicit hard and soft constraints over states, actions, and state features from demonstrations in deterministic and non-deterministic environments modeled as Markov Decision Processes (MDPs). Our method enables agents implicitly learn human constraints and desires without the need for explicit modeling by the agent designer and to transfer these constraints between environments. Our novel method generalizes prior work which only considered deterministic hard constraints and achieves state of the art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑