面向移动操作的风险感知强化学习
Risk-Aware Reinforcement Learning for Mobile Manipulation
AI总结：
本文提出了一种基于风险感知的强化学习方法，通过分布强化学习和模仿学习，实现移动操作中的风险敏感决策与策略蒸馏。
摘要：
为使机器人能够从实验室环境顺利过渡到日常环境，它们必须开始对自身行为的风险进行推理，并做出有意识的、风险感知的决策。这在执行移动操作任务的机器人中尤为重要，这些任务涉及与动态、无结构空间的交互和导航。然而，现有的移动机械臂整体控制器通常缺乏在不确定性下进行风险敏感决策的显式机制。到目前为止，我们首次（i）学习了基于以自身为中心的深度观察的视觉-运动策略，该策略在运行时可调整风险敏感性；（ii）展示了通过模仿学习（IL）可以将风险感知行为转移到基于以自身为中心的深度观察的视觉-运动策略中。我们的方法通过首先使用分布强化学习（DRL）训练一个特权教师策略，该策略使用风险中性分布批评者，然后将畸变风险度量应用于批评者的预测回报分布，以计算用于策略更新的风险调整优势估计，从而实现一系列风险感知行为。我们然后通过模仿学习蒸馏教师策略，获得基于以自身为中心的深度观察的风险感知学生策略。我们进行了广泛的评估，证明我们的训练视觉-运动策略在未映射环境中执行反应性整体运动时，表现出风险感知行为（特别是实现了更好的最坏情况性能），并利用实时深度观察进行感知。
英文摘要：
For robots to successfully transition from lab settings to everyday environments, they must begin to reason about the risks associated with their actions and make informed, risk-aware decisions. This is particularly true for robots performing mobile manipulation tasks, which involve both interacting with and navigating within dynamic, unstructured spaces. However, existing whole-body controllers for mobile manipulators typically lack explicit mechanisms for risk-sensitive decision-making under uncertainty. To our knowledge, we are the first to (i) learn risk-aware visuomotor policies for mobile manipulation conditioned on egocentric depth observations with runtime-adjustable risk sensitivity, and (ii) show risk-aware behaviours can be transferred through Imitation Learning (IL) to a visuomotor policy conditioned on egocentric depth observations. Our method achieves this by first training a privileged teacher policy using Distributional Reinforcement Learning (DRL), with a risk-neutral distributional critic. Distortion risk-metrics are then applied to the critic's predicted return distribution to calculate risk-adjusted advantage estimates used in policy updates to achieve a range of risk-aware behaviours. We then distil teacher policies with IL to obtain risk-aware student policies conditioned on egocentric depth observations. We perform extensive evaluations demonstrating that our trained visuomotor policies exhibit risk-aware behaviour (specifically achieving better worst-case performance) while performing reactive whole-body motions in unmapped environments, leveraging live depth observations for perception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
