Chain-of-Goals 分层策略用于长视界离线目标条件强化学习
Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL
AI总结：
提出 Chain-of-Goals 分层策略 (CoGHP)，通过自回归生成潜在子目标序列，利用 MLP-Mixer 统一架构建模层次决策，在长视界任务中优于离线基线方法。
摘要：
离线目标条件强化学习在长视界任务中仍然具有挑战性。虽然分层方法通过分解任务缓解了这一问题，但大多数现有方法依赖独立的高层和低层网络，且仅生成单个中间子目标，在长视界决策中留下若干结构限制。为解决这一限制，我们从思维链推理中汲取灵感，提出 Chain-of-Goals 分层策略 (CoGHP)，一种新颖的框架，将分层决策重新表述为统一架构内的自回归序列建模。给定状态和最终目标，CoGHP 自回归生成一系列潜在子目标，随后是原始动作，其中每个潜在子目标充当推理步骤，条件化后续预测。为高效实现这一点，我们引入 MLP-Mixer 骨干网络，支持跨令牌通信并捕获状态、目标、潜在子目标和动作之间的结构关系。在具有挑战性的导航和操作基准测试中，CoGHP 一致优于强离线基线，在长视界任务上展现出改进的性能。项目页面：此 https URL
英文摘要：
Offline goal-conditioned reinforcement learning remains challenging for long-horizon tasks. While hierarchical approaches mitigate this issue by decomposing tasks, most existing methods rely on separate high- and low-level networks and generate only a single intermediate subgoal, leaving several structural limitations in long-horizon decision-making. To address this limitation, we draw inspiration from chain-of-thought reasoning and propose the Chain-of-Goals Hierarchical Policy (CoGHP), a novel framework that reformulates hierarchical decision-making as autoregressive sequence modeling within a unified architecture. Given a state and a final goal, CoGHP autoregressively generates a sequence of latent subgoals followed by the primitive action, where each latent subgoal acts as a reasoning step that conditions subsequent predictions. To implement this efficiently, we introduce an MLP-Mixer backbone, which supports cross-token communication and captures structural relationships among state, goal, latent subgoals, and action. Across challenging navigation and manipulation benchmarks, CoGHP consistently outperforms strong offline baselines, demonstrating improved performance on long-horizon tasks. Project page: https://wlsdn9350.github.io/projects/coghp/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
