发表机构
Xi’an Jiaotong University（西安交通大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
CoDrift是面向离线强化学习的组合框架，通过组合三类目标场生成策略，在OGBench和D4RL的73个任务中，于离线和离线转在线设置均取得最优平均排名。
AI 中文摘要
离线强化学习本质上是多目标的：策略必须与固定数据集的行为支持保持兼容，同时优先选择高价值动作。我们通过将每个目标视为指定生成动作应如何移动的动作空间运动场，将这些目标重新表述为统一形式。该视角使异构学习目标能够通过场组合直接结合。受漂移模型启发，我们提出CoDrift，这是一种用于单步生成式策略学习的组合框架。CoDrift将三个目标级场组合为统一策略场：条件场保留依赖状态的行为结构，边缘场汇集跨状态的动作，以在连续控制离线强化学习的单正样本 regime 中提供更稳定的生成信号，价值场将生成动作推向更高价值区域。组合后的场被融入随机生成器，在部署时通过单次前向传播生成动作。我们在OGBench和D4RL的73个任务上，分别在离线和离线转在线设置中评估CoDrift。CoDrift与最先进方法相比表现出色，在两种设置中均取得最佳平均排名。
英文摘要
Offline reinforcement learning is intrinsically multi-objective: a policy must remain compatible with the behavioral support of a fixed dataset while preferentially selecting high-value actions. We recast these objectives in a common form by viewing each as an action-space motion field that specifies how generated actions should move. This perspective enables heterogeneous learning objectives to be combined directly through field composition. Inspired by drifting models, we propose CoDrift, a compositional framework for one-step generative policy learning. CoDrift combines three objective-level fields into a unified policy field. The conditional field preserves state-dependent behavioral structure, while the marginal field pools actions across states to provide a more stable generative signal in the single-positive-sample regime of continuous-control offline RL. The value field moves generated actions toward higher-value regions. The composed field is absorbed into a stochastic generator that produces an action with a single forward pass at deployment. We evaluate CoDrift on 73 tasks from OGBench and D4RL in both offline and offline-to-online settings. CoDrift compares favorably with state-of-the-art methods and achieves the best average rank in both settings.
