任务最优性约束下从不完美演示中学习风格
Constrained Style Learning from Imperfect Demonstrations under Task Optimality
AI总结：
本文针对不完美演示下风格学习与任务性能难以兼顾的问题，提出基于约束马尔可夫决策过程和自适应拉格朗日乘子的方法，实现高保真风格模仿与接近最优任务性能的平衡，并在多平台及 ANYmal-D 硬件上验证了有效性。
摘要：
从演示中学习已被证明在机器人学中能有效获取自然行为，例如风格化运动和逼真的敏捷性，尤其是在显式定义面向风格的奖励函数具有挑战性的情况下。为现实世界任务合成风格化运动通常需要在任务性能和模仿质量之间取得平衡。现有方法通常依赖与任务目标紧密对齐的专家演示。然而，实际演示往往不完整或不切实际，导致当前方法以牺牲任务性能为代价来提升风格。为了解决这一问题，我们提出将该问题形式化为一个约束马尔可夫决策过程（CMDP）。具体而言，我们优化一个风格模仿目标，并施加约束以维持接近最优的任务性能。我们引入了一个自适应可调的拉格朗日乘子，引导智能体有选择地模仿演示，在不损害任务性能的前提下捕捉风格细节。我们在多个机器人平台和任务上验证了我们的方法，展示了鲁棒的任务性能和高保真的风格学习。在 ANYmal-D 硬件上，我们展示了机械能降低 14.5% 以及更敏捷的步态模式，体现了现实世界的收益。
英文摘要：
Learning from demonstration has proven effective in robotics for acquiring natural behaviors, such as stylistic motions and lifelike agility, particularly when explicitly defining style-oriented reward functions is challenging. Synthesizing stylistic motions for real-world tasks usually requires balancing task performance and imitation quality. Existing methods generally depend on expert demonstrations closely aligned with task objectives. However, practical demonstrations are often incomplete or unrealistic, causing current methods to boost style at the expense of task performance. To address this issue, we propose formulating the problem as a constrained Markov Decision Process (CMDP). Specifically, we optimize a style-imitation objective with constraints to maintain near-optimal task performance. We introduce an adaptively adjustable Lagrangian multiplier to guide the agent to imitate demonstrations selectively, capturing stylistic nuances without compromising task performance. We validate our approach across multiple robotic platforms and tasks, demonstrating both robust task performance and high-fidelity style learning. On ANYmal-D hardware we show a 14.5% drop in mechanical energy and a more agile gait pattern, showcasing real-world benefits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
