CausalGDP: 基于因果引导的扩散策略用于强化学习
CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning
AI总结：
CausalGDP通过整合因果推理，提升基于扩散的强化学习策略在复杂高维控制任务中的性能。
摘要：
强化学习（RL）在广泛的序列决策问题中取得了显著的成功。最近的基于扩散的策略进一步通过建模复杂的、高维的动作分布来提高RL的效果。然而，现有的扩散策略主要依赖于统计关联，未能明确考虑状态、动作和奖励之间的因果关系，限制了其识别真正导致高回报的动作组件的能力。在本文中，我们提出了因果引导的扩散策略（CausalGDP），这是一个将因果推理整合到基于扩散的RL中的统一框架。CausalGDP首先从离线数据中学习一个基础的扩散策略和初始的因果动态模型，捕捉状态、动作和奖励之间的因果依赖关系。在实时交互过程中，因果信息被持续更新并作为指导信号融入，以引导扩散过程朝向那些因果影响未来状态和奖励的动作。通过超越关联而考虑因果性，CausalGDP将策略优化集中在真正推动性能提升的动作组件上。实验结果表明，CausalGDP在复杂、高维控制任务中 consistently 获得与最先进基于扩散的和离线RL方法相当或更优的表现。
英文摘要：
Reinforcement learning (RL) has achieved remarkable success in a wide range of sequential decision-making problems. Recent diffusion-based policies further improve RL by modeling complex, high-dimensional action distributions. However, existing diffusion policies primarily rely on statistical associations and fail to explicitly account for causal relationships among states, actions, and rewards, limiting their ability to identify which action components truly cause high returns. In this paper, we propose Causality-guided Diffusion Policy (CausalGDP), a unified framework that integrates causal reasoning into diffusion-based RL. CausalGDP first learns a base diffusion policy and an initial causal dynamical model from offline data, capturing causal dependencies among states, actions, and rewards. During real-time interaction, the causal information is continuously updated and incorporated as a guidance signal to steer the diffusion process toward actions that causally influence future states and rewards. By explicitly considering causality beyond association, CausalGDP focuses policy optimization on action components that genuinely drive performance improvements. Experimental results demonstrate that CausalGDP consistently achieves competitive or superior performance over state-of-the-art diffusion-based and offline RL methods, especially in complex, high-dimensional control tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
