离线安全强化学习的约束自适应策略切换
Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learning
AI总结：
提出约束自适应策略切换（CAPS）框架，通过在训练时学习多个不同奖励与成本权衡的策略，并在测试时根据当前成本约束动态切换策略，解决了离线安全强化学习中无需重训即可适应变化安全约束的挑战。
摘要：
离线安全强化学习（OSRL）旨在从固定的训练数据批次中学习决策策略，以最大化奖励并满足预定义的安全约束。然而，在部署期间无需重新训练即可适应不断变化的安全约束，仍是一个尚未充分探索的挑战。为应对此挑战，我们引入了约束自适应策略切换（CAPS），这是一个包裹现有离线RL算法的框架。在训练阶段，CAPS利用离线数据学习具有共享表示的多个策略，这些策略优化不同的奖励与成本权衡。在测试阶段，CAPS通过在每个状态下，从满足当前成本约束的策略中选择能最大化未来奖励的策略，从而在这些策略之间进行切换。我们在DSRL基准的38个任务上的实验表明，CAPS始终优于现有方法，为OSRL建立了一个强大的基于包裹器的基线。代码已在https://github.com/yassineCh/CAPS公开发布。
英文摘要：
Offline safe reinforcement learning (OSRL) involves learning a decision-making policy to maximize rewards from a fixed batch of training data to satisfy pre-defined safety constraints. However, adapting to varying safety constraints during deployment without retraining remains an under-explored challenge. To address this challenge, we introduce constraint-adaptive policy switching (CAPS), a wrapper framework around existing offline RL algorithms. During training, CAPS uses offline data to learn multiple policies with a shared representation that optimize different reward and cost trade-offs. During testing, CAPS switches between those policies by selecting at each state the policy that maximizes future rewards among those that satisfy the current cost constraint. Our experiments on 38 tasks from the DSRL benchmark demonstrate that CAPS consistently outperforms existing methods, establishing a strong wrapper-based baseline for OSRL. The code is publicly available at https://github.com/yassineCh/CAPS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
