在鲁棒风格对齐下高质量行为的离线强化学习
Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignment
AI总结：
本文提出SCIQL方法，通过结合离线目标条件RL技术与新的门控优势加权回归机制，在保持风格对齐的同时提升任务性能。
摘要：
我们研究了通过显式风格监督进行风格条件策略的离线强化学习，使用子轨迹标签函数。在这种设定下，对齐风格与高任务性能尤其具有挑战性，因为存在分布偏移和风格与奖励之间的内在冲突。现有方法尽管引入了众多风格定义，但往往无法有效协调这些目标。为了解决这些挑战，我们提出了一种统一的行为风格定义，并将其实例化为一个实用框架。基于此，我们引入了风格条件隐式Q学习（SCIQL），该方法利用了离线目标条件RL技术，如 hindsight relabeling 和 value learning，并结合新的 Gated Advantage Weighted Regression 机制，以高效优化任务性能同时保持风格对齐。实验表明，SCIQL 在两个目标上都优于先前的离线方法。代码、数据集和可视化可在：https://mathieu-petitbois.github.io/projects/sciql/ 获取。
英文摘要：
We study offline reinforcement learning of style-conditioned policies using explicit style supervision via subtrajectory labeling functions. In this setting, aligning style with high task performance is particularly challenging due to distribution shift and inherent conflicts between style and reward. Existing methods, despite introducing numerous definitions of style, often fail to reconcile these objectives effectively. To address these challenges, we propose a unified definition of behavior style and instantiate it into a practical framework. Building on this, we introduce Style-Conditioned Implicit Q-Learning (SCIQL), which leverages offline goal-conditioned RL techniques, such as hindsight relabeling and value learning, and combine it with a new Gated Advantage Weighted Regression mechanism to efficiently optimize task performance while preserving style alignment. Experiments demonstrate that SCIQL achieves superior performance on both objectives compared to prior offline methods. Code, datasets and visuals are available in: https://mathieu-petitbois.github.io/projects/sciql/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
