通过STL引导的Stein变分策略梯度从稀疏成功信号中学习机器人策略
Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient
- University of Pennsylvania（宾夕法尼亚大学）
- Lehigh University（里海大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出STL-SVPG方法，利用平滑STL鲁棒性作为轨迹级目标，从稀疏成功信号中学习机器人策略，在六个基准中的五个上取得最高平均成功率，并实现仿真到现实的迁移。
AI中文摘要：
在任务完成依赖于协调动作、精确接触结果或同时满足多个条件时，从稀疏成功信号中学习机器人策略具有挑战性。与世界的复杂物理交互进一步加剧了这些要求。先前使用传统奖励塑形机制的工作提供了密集反馈，但局部进展可能不会转化为最终任务完成。我们提出了信号时序逻辑引导的Stein变分策略梯度（STL-SVPG），一种基于种群的方法，使用平滑的STL鲁棒性作为轨迹级训练目标。通过动力学对该目标进行微分，根据策略动作对完整任务规范的影响（而非仅局部进展）为其分配信用。我们在六个四旋翼和机械臂任务上评估了该方法，这些任务涉及事件触发响应、严格有序行为、在指定截止日期内的响应以及与世界的物理交互。STL-SVPG在六个基准中的五个上取得了比较方法中最高的平均成功率。在仿真中训练的策略将时序和接触任务行为迁移到现实世界。
英文摘要：
Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditions together. Intricate physical interactions with the world further complicate these requirements. Prior work using conventional reward shaping mechanisms provides dense feedback but local progress might not translate into eventual task completion. We present Signal Temporal Logic-guided Stein Variational Policy Gradient (STL-SVPG), a population-based method that uses smooth STL robustness as a trajectory-level training objective. Differentiating this objective through the dynamics assigns credit to policy actions according to their effect on the complete task specification, rather than local progress alone. We evaluate the approach on six quadcopter and manipulator tasks that involves event-triggered responses, strictly ordered behavior, responses within specified deadlines, and physical interaction with the world. STL-SVPG achieves the highest mean success rate among the compared methods on five of six benchmarks. Simulation-trained policies trained in simulation transfer temporal and contact task behavior to the real world.
