This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds.
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Université de Montréal（蒙特利尔大学）；Mila - Québec AI Institute（蒙特利尔人工智能研究所）；Université Laval（拉瓦尔大学）；IID - Institut Intelligence et Données（智能与数据研究所）；LawZero；CIFAR AI Chair（CIFAR人工智能主席）
Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization
Bosch Corporate Research（博世公司研究中心）；Technical University Darmstadt（达姆施塔特工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science（认知科学中心）
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
DeepMind（深度思维）；University of Oxford（牛津大学）
使用环境模型和价值函数，智能体可以通过将模型展开不同的长度并使用其价值函数进行自举来构建状态值的许多估计。我们的主要观点是，可以将这组值估计视为一种集成，我们称之为\emph{implicit value ensemble}（IVE）。因此，这些估计之间的差异可以用作代理的认知不确定性;我们将此信号称为\emph{model-value inconsistency}或\emph{self-inconsistency}。与通过训练许多模型和/或价值函数的集合来估计不确定性的现有工作不同，这种方法仅需要在大多数基于基于模型的强化学习算法中已经学习的单个模型和价值函数。我们提供了经验证据，在表格和函数近似设置从像素的自我不一致性是有用的（i）作为一个信号的探索，（ii）安全地采取行动下分布偏移，和（iii）鲁棒基于值的规划与学习模型。
Learning Soft Robotic Dynamics with Active Exploration
Soft Robotics Lab, D-MAVT, ETH Zurich（软机器人实验室，ETH Zurich）；Learning & Adaptive Systems Group, D-INFK, ETH Zurich（学习与自适应系统组，ETH Zurich）；ETH AI Center, ETH Zurich（ETH人工智能中心，ETH Zurich）
Towards Generalizable Visually Grounded Exploration of Household Devices
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology（哈尔滨工业大学社会计算与交互机器人研究中心）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
FlagVNE: A Flexible and Generalizable Reinforcement Learning Framework for Network Resource Allocation
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件学院）；Guangzhou HKUST Fok Ying Tung Research Institute（香港科技大学广州霍英东研究院）；School of Artificial Intelligence, Peking University（北京大学人工智能学院）；Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能学域）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程学系）；Microsoft（微软公司）
GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与电子工程学院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
