PROGRESSOR：一种具有自监督在线优化的感知引导奖励估计器
PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinement
AI总结：
针对无人工监督的目标条件强化学习策略训练问题，提出PROGRESSOR框架，通过自监督学习任务进度分布估计奖励函数，在线RL训练中对抗性优化以缓解分布偏移，预训练后无需微调即可泛化到真实机器人任务，性能优于现有方法。
摘要：
我们提出PROGRESSOR，这是一种从视频中学习任务无关奖励函数的新型框架，可通过目标条件强化学习（RL）实现策略训练，无需人工监督。该奖励的核心是任务进度分布的估计，该估计是当前、初始和目标观测的函数，以自监督方式学习。关键在于，PROGRESSOR在在线RL训练期间通过对分布外观测的预测进行反向调整，对抗性地优化奖励，以缓解非专家观测中固有的分布偏移。将这种进度预测作为密集奖励并结合对抗性反向调整，我们证明PROGRESSOR能使机器人在无外部监督的情况下学习复杂行为。在EPIC-KITCHENS的大规模第一人称人类视频上预训练后，PROGRESSOR无需针对域内任务特定数据进行微调，即可泛化到嘈杂演示下的真实机器人离线RL，性能优于为机器人学习提供密集视觉奖励的现有方法。我们的研究结果突显了PROGRESSOR在无法轻易获得直接动作标签和任务特定奖励的可扩展机器人应用中的潜力。
英文摘要：
We present PROGRESSOR, a novel framework that learns a task-agnostic reward function from videos, enabling policy training through goal-conditioned reinforcement learning (RL) without manual supervision. Underlying this reward is an estimate of the distribution over task progress as a function of the current, initial, and goal observations that is learned in a self-supervised fashion. Crucially, PROGRESSOR refines rewards adversarially during online RL training by pushing back predictions for out-of-distribution observations, to mitigate distribution shift inherent in non-expert observations. Utilizing this progress prediction as a dense reward together with an adversarial push-back, we show that PROGRESSOR enables robots to learn complex behaviors without any external supervision. Pretrained on large-scale egocentric human video from EPIC-KITCHENS, PROGRESSOR requires no fine-tuning on in-domain task-specific data for generalization to real-robot offline RL under noisy demonstrations, outperforming contemporary methods that provide dense visual reward for robotic learning. Our findings highlight the potential of PROGRESSOR for scalable robotic applications where direct action labels and task-specific rewards are not readily available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
