当传感器失效时：用于在传感器漂移下鲁棒PPO的时序序列模型
When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift
AI总结：
本文研究了在传感器漂移下PPO的鲁棒性，通过引入时序序列模型提升政策在部分可观测性和表示偏移下的性能。
摘要：
现实中的强化学习系统必须在其观测流中应对分布漂移，但大多数策略架构隐式假设完全观测且无噪声的状态。我们研究了在时间持续的传感器故障下PPO的鲁棒性，这些故障导致部分可观测性和表示偏移。为应对这种漂移，我们增强了PPO，加入Transformer和状态空间模型（SSMs）等时序序列模型，使策略能从历史中推断缺失信息并保持性能。在随机传感器故障过程中，我们证明了一个高概率的无限时间奖励退化界，量化了鲁棒性如何依赖于策略的平滑性和故障持续性。经验上，在具有严重传感器丢失的MuJoCo连续控制基准上，我们发现基于Transformer的序列策略在鲁棒性上显著优于MLP、RNN和SSM基线，在大量传感器不可用时仍能保持高回报。这些结果表明，时序序列推理提供了一种原则性和实用的机制，以在由传感器不可靠引起的观测漂移下实现可靠运行。
英文摘要：
Real-world reinforcement learning systems must operate under distributional drift in their observation streams, yet most policy architectures implicitly assume fully observed and noise-free states. We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures that induce partial observability and representation shift. To respond to this drift, we augment PPO with temporal sequence models, including Transformers and State Space Models (SSMs), to enable policies to infer missing information from history and maintain performance. Under a stochastic sensor failure process, we prove a high-probability bound on infinite-horizon reward degradation that quantifies how robustness depends on policy smoothness and failure persistence. Empirically, on MuJoCo continuous-control benchmarks with severe sensor dropout, we show Transformer-based sequence policies substantially outperform MLP, RNN, and SSM baselines in robustness, maintaining high returns even when large fractions of sensors are unavailable. These results demonstrate that temporal sequence reasoning provides a principled and practical mechanism for reliable operation under observation drift caused by sensor unreliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
