世界模型与现实对齐的不可约量子优势

中文摘要 AI 辅助

世界模型提供真实世界的数字模拟，使智能体在成本高昂的现实部署前即可接受训练与测试。每个时间步，世界模型接收一个动作，生成与真实世界统计特性匹配的观测和奖励。在当前结果依赖于久远事件的复杂环境中，这需要记忆。人们可能期望，通过增加记忆，总能构建一个足够准确的模型，使真实世界与虚拟世界的最优智能体策略对齐。我们证明，对于经典世界模型，这是错误的，即使真实世界本身是经典的。我们构造了一些真实世界，对于这些世界，每个有限经典模型在同一条可能轨迹上都会失效：要么在真实世界明确偏好某一动作时失去区分动作的能力，要么反复将最高期望奖励分配给次优动作。其期望奖励估计值也存在非零的平均误差。相比之下，对于每个这样的真实世界，存在一个使用单个qutrit（三能级量子位）的量子世界模型，可精确重现该真实世界：其奖励估计值与偏好动作始终与真实世界匹配，确保真实与虚拟世界的最优策略保持完美对齐。

英文摘要

World models provide digital simulacra of the true world, allowing agents to be trained and tested before costly real-world deployment. At each time step, they receive an action and generate an observation and reward matching the statistics of the true world. In complex environments where present outcomes depend on events far in the past, this requires memory. One might expect that, by increasing memory, we can always build a model accurately enough to align the optimal agent policies of the real and virtual worlds. We show that this is false for classical world models, even when the true world itself is classical. We construct true worlds for which every finite classical model fails along the same possible trajectory: it either loses the ability to distinguish actions when the true world clearly prefers one, or repeatedly assigns the highest expected reward to suboptimal actions. Its expected-reward estimates also retain a nonvanishing average error. In contrast, each such true world admits a quantum world model using a single qutrit that reproduces it exactly: its reward estimates and preferred actions always match those of the true world, ensuring that the optimal policies of the real and virtual worlds remain perfectly aligned.

发表机构

• Nanyang Technological University（南洋理工大学）
• Centre for Quantum Technologies（量子技术中心）
• Nanyang Quantum Hub（南洋量子中心）
• School of Physical and Mathematical Sciences（物理与数学科学学院）
• College of Computing and Data Science（计算与数据科学学院）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

36 pages, 8 figures