基于单类决策树从演示中学习安全约束
Learning Safety Constraints From Demonstration Using One-Class Decision Trees
AI总结：
针对自主智能体安全对齐中人工定义奖励/成本函数易出错的问题，提出基于单类决策树从专家演示中学习可解释的析取范式安全约束，并结合约束强化学习获取安全策略，在合成基准和真实驾驶场景中验证了有效性。
摘要：
在物理环境中部署自主智能体时，智能体与人类价值观的对齐是一项核心挑战，此类场景下安全性是重要考量。然而，将智能体的目标定义为奖励和/或成本函数的做法本质上十分复杂，且容易出现人为错误。针对这一挑战，我们提出了一种利用单类决策树从专家演示中学习的新方法。这些决策树为将给定环境的相关约束表示为析取范式的逻辑公式提供了基础。学习到的约束随后被应用于预言机约束强化学习框架中，以获取安全策略。与其他方法相比，我们的方法可提供约束的可解释表示，这在安全关键型环境中是一项关键特性。为验证所提方法的有效性，我们在合成基准域和真实驾驶环境中开展了实验。
英文摘要：
The alignment of autonomous agents with human values is a pivotal challenge when deploying these agents within physical environments, where safety is an important concern. However, defining the agent's objective as a reward and/or cost function is inherently complex and prone to human errors. In response to this challenge, we present a novel approach that leverages one-class decision trees to facilitate learning from expert demonstrations. These decision trees provide a foundation for representing a set of constraints pertinent to the given environment as a logical formula in disjunctive normal form. The learned constraints are subsequently employed within an oracle constrained reinforcement learning framework, enabling the acquisition of a safe policy. In contrast to other methods, our approach offers an interpretable representation of the constraints, a vital feature in safety-critical environments. To validate the effectiveness of our proposed method, we conduct experiments in synthetic benchmark domains and a realistic driving environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
