用于离线轨迹规划的贝叶斯流网络

中文摘要 AI 辅助

离线强化学习（RL）利用静态数据集学习决策策略，无需实时与环境交互。尽管近期序列建模方法依赖连续扩散模型进行轨迹合成，但将这些方法应用于离散规划任务时，需要采用分类公式而非标准高斯构造。我们提出BFN-RL，这是一种基于贝叶斯流网络（BFNs）的离线RL统一生成建模框架。通过迭代更新分布参数而非噪声数据实例，BFN-RL可在单一概率公式内自然地对离散和连续轨迹空间进行建模。分类规划器生成未来状态序列，而学习到的逆动力学模型将连续生成的状态转换为动作。在离散规划和连续控制任务中的评估表明，BFN-RL能够在分类和连续状态空间中生成有效轨迹。我们的研究结果确立了BFNs作为适用于跨数据模态的离线轨迹规划的通用生成基础的地位。

英文摘要

Offline reinforcement learning (RL) leverages static datasets to learn decision policies without real-time environment interaction. While recent sequence-modeling approaches rely on continuous diffusion models for trajectory synthesis, applying these methods to discrete planning tasks requires a categorical formulation rather than the standard Gaussian construction. We present BFN-RL, a unified generative modeling framework for offline RL based on Bayesian Flow Networks (BFNs). By iteratively evolving distribution parameters rather than noisy data instances, BFN-RL natively models both discrete and continuous trajectory spaces within a single probabilistic formulation. The categorical planner generates future state sequences, and a learned inverse-dynamics model converts consecutive generated states into actions. Evaluations in discrete planning and continuous control show that BFN-RL can generate effective trajectories across both categorical and continuous state spaces. Our results establish BFNs as a versatile generative foundation for offline trajectory planning across data modalities.

发表机构

• Norwegian University of Science and Technology（挪威科技大学）

机构由 AI 辅助整理，请以论文原文为准。