arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06951cs.AIcs.LGcs.MA
收藏

多目标约束推断使用逆强化学习

Multi-Objective Constraint Inference using Inverse reinforcement learning

Faculty of Sciences(科学学院)

Syed Ihtesham Hussain Shah, Floris den Hengst, Aneta Lisowska, Annette ten Teije

AI总结:

本文提出多目标约束推断框架MOCI,用于从异质专家轨迹中联合提取共享约束和个体偏好,有效处理不同目标下的多样且可能冲突的行为,实验证明其在预测性能和计算效率方面优于现有方法。

摘要:

约束推断被广泛认为是通过观察专家示范将强化学习代理对齐安全边界和操作指南的关键。然而,现有方法通常假设示范是同质的(即由单一专家或目标相同的多个专家生成)。它们还难以捕捉个体偏好,且常面临计算效率低下的问题。本文引入多目标约束推断(MOCI),一种新的框架,旨在从异质专家轨迹中联合提取共享约束和个体偏好,其中多个专家追求不同目标。MOCI有效建模和学习多样且可能冲突的行为。实验证明,MOCI在预测性能上显著优于现有基线方法,并在标准网格世界基准上保持竞争性的计算效率。这些结果确立了MOCI作为准确、灵活且计算可行的方法,用于现实中的约束推断和偏好学习任务。

英文摘要:

Constraint inference is widely considered essential to align reinforcement learning agents with safety boundaries and operational guidelines by observing expert demonstrations. However, existing approaches typically assume homogeneous demonstrations (i.e., generated by a single expert or multiple experts with identical objectives). They also have limited ability to capture individual preferences and often suffer from computational inefficiencies. In this paper, we introduce Multi-Objective Constraint Inference (MOCI), a novel framework designed to jointly extract shared constraints and individual preferences from heterogeneous expert trajectories, where multiple experts pursue different objectives. MOCI effectively models and learns from diverse, and potentially conflicting, behaviors. Empirical evaluations demonstrate that MOCI significantly outperforms existing baselines, achieving improved predictive performance, and maintaining competitive computational efficiency on a standard grid-world benchmark. These results establish MOCI as an accurate, flexible, and computationally practical approach for real-world constraint inference and preference learning tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑