Open-world novelty--a sudden change in the mechanics or properties of an environment--is a common occurrence in the real world. Novelty adaptation is an agent's ability to improve its policy performance post-novelty. Most reinforcement learning (RL) methods assume that the world is a closed, fixed process. Consequentially, RL policies adapt inefficiently to novelties. To address this, we introduce WorldCloner, an end-to-end trainable neuro-symbolic world model for rapid novelty adaptation. WorldCloner learns an efficient symbolic representation of the pre-novelty environment transitions, and uses this transition model to detect novelty and efficiently adapt to novelty in a single-shot fashion. Additionally, WorldCloner augments the policy learning process using imagination-based adaptation, where the world model simulates transitions of the post-novelty environment to help the policy adapt. By blending ''imagined'' transitions with interactions in the post-novelty environment, performance can be recovered with fewer total environment interactions. Using environments designed for studying novelty in sequential decision-making problems, we show that the symbolic world model helps its neural policy adapt more efficiently than model-based and model-based neural-only reinforcement learning methods.
HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；Centre for Cognitive Science of Darmstadt（达姆施塔特认知科学中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
ANNEAL: Adapting LLM Agents via Governed Symbolic Patch Learning
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；University at Buffalo（布法罗大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Colorado Colorado Springs（科罗拉多大学科罗拉多州立分校）
Novelty-organizing team of classifiers in noisy and dynamic environments
Graduate School of Information Science（信息科学研究生学校）；Electrical Engineering Kyushu University Fukuoka, Japan Email（电气工程九州大学福冈日本电子邮件）；Faculty of Information Science（信息科学学院）
Novelty Accommodating Multi-Agent Planning in High Fidelity Simulated Open World
Naval Information Warfare Center Pacific（海军信息作战中心太平洋分部）；SRI International（SRI国际）；Ben Gurion University（本·古里安大学）；Calspan-University of Buffalo Research Center（卡尔span-布法罗大学研究中心）
From One-Shot Generation to Incremental Music Composition: Adapting a General-Purpose Instruction LLM for Persistent Symbolic Editing
Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio)（里约热内卢天主教大学）；Sorbonne Université（索邦大学）；LIP6（巴黎第六大学计算机科学实验室）；Universidade Federal do Rio de Janeiro (UFRJ)（里约热内卢联邦大学）
