环境群胚的范畴化内部化用于可泛化POMDP求解
Categorical Internalisation of Environmental Groupoids for Generalisable POMDP Solving
浏览论文内容
中文总结 AI 辅助
本文利用范畴论将环境对称性建模为群胚，通过轨道划分缩减状态空间，提升部分可观测强化学习的样本效率与泛化性能。
中文摘要 AI 辅助
本文倡导将范畴论作为在高维、部分可观测环境中构建和改进强化学习的实用框架。我们通过将状态空间划分为由对称轨道诱导的等价类来建模环境状态之间的对称性，并将每个这样的类组织为一个具有指定典型代表的群胚。这使得智能体能够同时在许多相似的环境状态之间共享所学知识，而不是将每个方向或位置视为全新的问题。因此，学习在对称缩减的状态空间上进行，每个轨道仅表示一次，在保留结构的同时消除冗余并提高样本效率。我们在标准强化学习流程中实现此框架，并在部分可观测基准上评估两种不同方法，证明在具有潜在对称性的环境中，基于轨道的划分能带来一致的性能提升。除了这些实证结果，我们的方法还展示了范畴结构如何在抽象强化学习公式与其计算应用之间提供原则性的桥梁，从而为构建更结构化、可扩展的学习系统开辟了道路。
英文摘要
This paper advocates category theory as a practical framework for structuring and improving rein- forcement learning in high-dimensional, partially observable environments. We model symmetries between environmental states by partitioning the state space into equivalence classes induced by sym- metry orbits, and organise each such class as a groupoid with a designated canonical representative. This allows the agent to share what it learns across many similar environmental states simultaneously, rather than treating every orientation or position as an entirely new problem. Learning is thus carried out on a symmetry-reduced state space with each orbit represented once, preserving structure while eliminating redundancy and improving sample efficiency. We implement this framework within standard reinforcement learning pipelines and evaluate two different approaches on partially observable benchmarks, demonstrating that orbit-based partitioning yields consistent performance improvements in environments exhibiting latent symmetry. Beyond these empirical results, our approach illustrates how categorical structure provides a principled bridge between abstract reinforcement learning formulations and their computational application, thereby establishing a pathway toward more structured and scalable learning systems.
发表机构
- City St George’s, University of London（伦敦大学城市圣乔治学院）
机构由 AI 辅助整理，请以论文原文为准。
