UniIntervene：用于高效现实世界强化学习的智能干预
UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning
AI总结：
提出UniIntervene智能干预模型，通过检测低效探索并自主恢复策略至高价值状态，在真实机器人操作任务中平均成功率提升8.6%，人类干预减少57%。
摘要：
人在回路强化学习（HiL-RL）已成为现实世界机器人操作的有效范式，能够通过人类指导实现在线策略改进。然而，当前的HiL-RL框架仍然依赖频繁的人类干预来纠正策略，使其脱离低效探索，这导致高昂的人力成本并限制了现实世界的可扩展性。为解决这一问题，我们提出UniIntervene，一种智能干预模型，它能够检测低效探索并自主将策略恢复至高价值状态，从而接管人类操作员的大部分干预工作。具体而言，UniIntervene首先执行未来条件化的动作价值估计，预测当前动作的潜在后果并评估其诱导价值，从而提供更稳定的进展信号。在此基础上，一个时间价值风险评论家聚合最近的价值动态，并在估计价值出现持续停滞或下降时触发干预。当需要干预时，UniIntervene从过去干预事件的内存中检索高价值恢复目标，并通过目标条件化的恢复策略生成可执行的纠正动作。通过这种方式，UniIntervene将干预从被动的人类纠正转变为价值感知的恢复过程，从而实现高效的现实世界强化学习。在多种真实世界操作任务上的大量实验表明，与最先进的HiL-RL基线相比，UniIntervene将平均成功率提高了8.6%，同时将人类干预减少了57%。
英文摘要：
Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real-world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators. Specifically, UniIntervene first performs future-conditioned action-value estimation, predicting the latent consequence of the current action and evaluating its induced value, which provides a more stable progress signal. Building on this, a temporal value-risk critic aggregates recent value dynamics and triggers intervention when the estimated value exhibits sustained stagnation or degradation. When intervention is required, UniIntervene retrieves a high-value recovery target from a memory of past intervention episodes and produces executable corrective actions through a goal-conditioned recovery policy. In this way, UniIntervene turns intervention from passive human correction into a value-aware recovery process for efficient real-world RL. Extensive experiments on diverse real-world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
