arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

特权评论者训练实现端到端强化学习中无传感器推进器故障自适应

Privileged Critic Training Enables Sensor-Free Thruster Fault Adaptation in End-to-End RL

Ricard Marsal I Castan, Miguel A. Olivares-Méndez

arXiv 2608.22976首次发表:更新:

发表机构

University of Luxembourg(卢森堡大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究提出RAFT策略,通过特权评论者训练实现无传感器推进器故障自适应,在浮动平台机器人多推进器故障场景下大幅提升容错导航成功率,缩小与神谕策略的性能差距。

AI 中文摘要

推进器驱动机器人的容错导航需要对非二元且不可完全观测的故障进行在线自适应:推进器可能持续退化、完全失效或卡滞开启。传统故障检测管道需要部署时无法获取的专用传感器;可观测真实故障状态的神谕控制器同样不切实际。我们表明,特权评论者训练足以实现无传感器故障自适应:在训练期间让PPO(近端策略优化)价值函数访问真实退化状态,而策略网络仅接收标准任务观测,以此塑造一种在部署时无需任何专用故障传感即可补偿故障的策略。我们提出RAFT(Recurrent Asymmetric Fault Tolerant,循环非对称容错),一种采用特权非对称评论者训练的带循环记忆的策略。在配备8个推进器、1个反作用轮的浮动平台机器人上,针对最多4个同时推进器故障的情况进行评估,RAFT在4个并发故障下的成功率达70.2%,缩小了从无故障基线(4.8%)到部署时可观测完整退化状态的神谕策略(82.4%)之间84%的差距。所有代码、检查点和数据均为开源。

英文摘要

Fault-tolerant navigation for thruster-actuated robots requires online adaptation to failures that are neither binary nor fully observable: thrusters may degrade continuously, fail dead, or jam stuck-open. Classical fault detection pipelines require dedicated sensors unavailable at deployment; oracle controllers that observe the true failure state are equally impractical. We show that privileged critic training is sufficient for sensor-free fault adaptation: giving the PPO value function access to the true degradation state dgt during training, while the actor receives only standard task observations, shapes a policy that compensates for failures at deployment without any dedicated fault sensing. We propose RAFT (Recurrent Asymmetric Fault Tolerant), a policy with recurrent memory trained with a privileged asymmetric critic. Evaluated on a floating-platform robot (8 thrusters, 1 reaction wheel) under up to four simultaneous thruster failures, RAFT achieves 70.2% success at four concurrent failures, closing 84% of the gap from a failure-naive baseline (4.8%) to an oracle policy that sees the full degradation state at deployment (82.4%). All code, checkpoints, and data are open-source.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑