学习主观时间尺度模型的情景记忆
Episodic Memory for Learning Subjective-Timescale Models
摘要：
在基于模型的学习中，智能体的模型通常是在环境的连续状态之间的转换上定义的，即使规划通常需要在多步时间尺度上进行推理，中间状态要么是不必要的，要么更糟，积累预测误差。相比之下，生物有机体中的智能行为的特征是根据上下文在不同的时间尺度上进行计划的能力。受人类时间感知最近的作品的启发，我们设计了一种新的方法来学习过渡动力学模型，基于定义代理的主观时间尺度的情景记忆的序列-在此期间，它学习世界动态和未来的规划进行。我们实现这一主动推理的框架，并证明由此产生的主观时间尺度模型（STM）可以系统地改变其预测的时间范围，同时保持相同的计算效率。此外，我们发现STM预测更有可能引入未来的显著事件（例如新物体进入视野），激励对环境新领域的探索。因此，STM产生更多信息的行动条件的推出，帮助代理做出更好的决策。我们验证了我们的STM代理的性能在动物AI环境中对基线系统的显着改善，使用环境的目标时标动态训练。
英文摘要：
In model-based learning, an agent's model is commonly defined over transitions between consecutive states of an environment even though planning often requires reasoning over multi-step timescales, with intermediate states either unnecessary, or worse, accumulating prediction error. In contrast, intelligent behaviour in biological organisms is characterised by the ability to plan over varying temporal scales depending on the context. Inspired by the recent works on human time perception, we devise a novel approach to learning a transition dynamics model, based on the sequences of episodic memories that define the agent's subjective timescale - over which it learns world dynamics and over which future planning is performed. We implement this in the framework of active inference and demonstrate that the resulting subjective-timescale model (STM) can systematically vary the temporal extent of its predictions while preserving the same computational efficiency. Additionally, we show that STM predictions are more likely to introduce future salient events (for example new objects coming into view), incentivising exploration of new areas of the environment. As a result, STM produces more informative action-conditioned roll-outs that assist the agent in making better decisions. We validate significant improvement in our STM agent's performance in the Animal-AI environment against a baseline system, trained using the environment's objective-timescale dynamics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
