基于潜在时间抽象的上下文规划
In-Context Planning with Latent Temporal Abstractions
AI总结：
I-TAP通过学习离散时间抽象空间，实现高效且鲁棒的上下文规划，在部分可观测和随机动态环境中表现优异。
摘要：
基于规划的强化学习在连续控制中受到两个实际问题的限制：在原始时间尺度上进行规划会导致指数级的分支和长时间跨度，而真实环境经常是部分可观测的，并且表现出制度转换，这会推翻静态、完全可观测的动力学假设。我们介绍了I-TAP（基于上下文的潜在时间抽象规划器），一种离线强化学习框架，它在学习的离散时间抽象空间中统一了上下文适应与在线规划。从离线轨迹中，I-TAP学习了一个观察条件的残差量化变分自编码器，将每个观察-宏动作片段压缩成一个从粗到细的离散残差标记堆栈，并一个时间Transformer自动回归预测这些标记堆栈，从短时间的历史中。所得到的序列模型同时充当一个基于上下文的抽象动作先验和潜在动力学模型。在测试时，I-TAP在标记空间中直接执行蒙特卡洛树搜索，使用短历史进行隐式适应而不进行梯度更新，并将选定的标记堆栈解码为可执行的动作。在确定性MuJoCo、随机性MuJoCo（带有每episode潜在动态制度）以及高维Adroit操作中，包括部分可观测变体，I-TAP始终一致地匹配或优于强大的无模型和有模型离线基线，证明了在随机动态和部分可观测性下的高效和鲁棒的上下文规划。
英文摘要：
Planning-based reinforcement learning for continuous control is bottlenecked by two practical issues: planning at primitive time scales leads to prohibitive branching and long horizons, while real environments are frequently partially observable and exhibit regime shifts that invalidate stationary, fully observed dynamics assumptions. We introduce I-TAP (In-Context Latent Temporal-Abstraction Planner), an offline RL framework that unifies in-context adaptation with online planning in a learned discrete temporal-abstraction space. From offline trajectories, I-TAP learns an observation-conditioned residual-quantization VAE that compresses each observation-macro-action segment into a coarse-to-fine stack of discrete residual tokens, and a temporal Transformer that autoregressively predicts these token stacks from a short recent history. The resulting sequence model acts simultaneously as a context-conditioned prior over abstract actions and a latent dynamics model. At test time, I-TAP performs Monte Carlo Tree Search directly in token space, using short histories for implicit adaptation without gradient update, and decodes selected token stacks into executable actions. Across deterministic MuJoCo, stochastic MuJoCo with per-episode latent dynamics regimes, and high-dimensional Adroit manipulation, including partially observable variants, I-TAP consistently matches or outperforms strong model-free and model-based offline baselines, demonstrating efficient and robust in-context planning under stochastic dynamics and partial observability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
