AI 中文总结
本文基于计算力学，将世界模型按建模信道分为三类，构建规范模型并通过POMDP示例说明耦合与支持限制可简化模型复杂度，为世界模型的本质提供了清晰框架。
AI 中文摘要
世界模型是基于模型的强化学习的核心组成部分，通常根据其预测的变量（如观测值、奖励、状态、隐状态或信息状态）进行讨论。本文提出存在一个更基础的区分维度：世界模型建模的信道。我们考虑三种情况：环境信道$O_{:} \mid A_{:}$、智能体信道$A_{:} \mid O_{:}$，以及已实现的联合过程$(A, O)_{:}$，其等价于无输入的信道。利用计算力学，我们将这三种情况的规范预测模型定义为$\epsilon$-换能器或$\epsilon$-机。规范环境模型可恢复标准预测状态表示，而另外两种情况则为智能体和联合系统提供了类似的规范模型概念。随后，我们构建由闭环耦合诱导的规范支持受限环境模型与智能体模型，其预测等价性覆盖已实现交互所支持的延续。关键结构结果表明，规范支持受限环境状态可通过规范联合因果状态分解，其转移结构直接由联合模型诱导；智能体侧的构建则是对偶的。最后，我们给出一个POMDP/控制器示例，其中无约束环境模型具有无穷多个状态，而由耦合诱导的规范支持受限模型是有限的。该框架阐明了不同世界模型所建模的对象，以及耦合和支持限制如何改变其规范预测结构与复杂度。
英文摘要
World models are a central component of model-based reinforcement learning. They are usually discussed in terms of what variables they predict, such as observations, rewards, states, latent or information states. We argue that there is a prior distinction: which channel they model. We consider three cases: the environment channel $O_{:} \mid A_{:}$, the agent channel $A_{:} \mid O_{:}$, and the realised joint process $(A, O)_{:}$, equivalently viewed as a channel with no inputs. Using computational mechanics, we define canonical predictive models for these three cases as $ε$-transducers or $ε$-machines. Canonical environment models recover standard predictive state representations, while the other two give analogous notions of canonical models for the agent and the joint system. We then build canonical support-restricted environment and agent models induced by closed-loop coupling, whose predictive equivalences range over continuations supported by the realised interaction. The key structural result is that canonical support-restricted environment states factor through the canonical joint causal states, and their transition structure is induced directly from the joint model; the agent-side construction is dual. Finally, we give a POMDP/controller example in which the unrestricted environment model has infinitely many states while the canonical support-restricted model induced by the coupling is finite. The framework clarifies what different world models are models of, and how coupling and support restriction can change their canonical predictive structure and complexity.
