In the present paper, we propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. Dreamer is a sample- and cost-efficient solution to robot learning, as it is used to train latent state-space models based on a variational autoencoder and to conduct policy optimization by latent trajectory imagination. However, this autoencoding based approach often causes object vanishing, in which the autoencoder fails to perceives key objects for solving control tasks, and thus significantly limiting Dreamer's potential. This work aims to relieve this Dreamer's bottleneck and enhance its performance by means of removing the decoder. For this purpose, we firstly derive a likelihood-free and InfoMax objective of contrastive learning from the evidence lower bound of Dreamer. Secondly, we incorporate two components, (i) independent linear dynamics and (ii) the random crop data augmentation, to the learning scheme so as to improve the training performance. In comparison to Dreamer and other recent model-free reinforcement learning methods, our newly devised Dreamer with InfoMax and without generative decoder (Dreaming) achieves the best scores on 5 difficult simulated robotics tasks, in which Dreamer suffers from object vanishing.
Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trust
National University of Singapore（新加坡国立大学）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）
Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds
Center for Brain-Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China（脑启发计算研究中心，精密仪器系，清华大学，北京，中国）；College of Computer Science and Technology, Zhejiang University, Hangzhou, China（计算机科学与技术学院，浙江大学，杭州，中国）；Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China（彭途萨微纳米科学与技术研究院，厦门大学，厦门，中国）
GIRL: Generative Imagination Reinforcement Learning via Information-Theoretic Hallucination Control
Department of Computer Science and Engineering, Visvesvaraya Technological University (VTU), Belagavi, India（维斯瓦拉亚科技大学计算机科学与工程系，贝拉加维，印度）；Aliens on Earth (AoE) Autonomous Research Group, Belagavi, India（地球外星人自主研究组，贝拉加维，印度）
Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination
Autonomous Intelligent Systems, Computer Science Institute VI - Intelligent Systems（自主智能系统，计算机科学研究所VI - 智能系统）；Robotics, Center for Robotics（机器人学，机器人中心）；the Lamarr Institute for Machine Learning（拉马尔机器学习研究所）；Artificial Intelligence, University of Bonn, Germany（人工智能，波恩大学，德国）
Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
Institute of Automation, CAS（中国科学院自动化研究所）；Chongqing Chang’an Technology Co., Ltd（重庆长安科技有限公司）；School of Advanced Interdisciplinary Sciences, UCAS（中国科学院大学先进交叉科学学院）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）
Predicting Consequences and Reinforcing Navigation Policies with Latent World Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Transferable Delay-Aware Reinforcement Learning via Implicit Causal Graph Modeling
College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室）；Institute of Military Transportation（军事交通运输研究院）；School of Artificial Intelligence, Hebei University of Technology（人工智能学院，河北工业大学）
Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination
College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）；School of Engineering, Newcastle University（纽卡斯尔大学工程学院）
World Models for Autonomous Navigation of Terrestrial Robots from LIDAR Observations
Universidade Federal de Santa Maria, Brazil（巴西联邦大学圣玛利亚分校）；University of Tsukuba, Japan（日本筑波大学）；Universidade Federal de Rio Grande（巴西联邦大学里约格兰德分校）；Universidad Tecnológica del Uruguay, Uruguay（乌拉圭技术大学）
One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability
New York University（纽约大学）；Independent Researcher（独立研究者）；Veermata Jijabai Technological Institute（韦尔玛塔·吉贾拜技术学院）；Mercity；University of Southern California（南加州大学）；Independent Researcher, MIT（麻省理工学院独立研究者）；Independent Researcher, GITAM（GITAM独立研究者）；Georgia Institute of Technology（佐治亚理工学院）
SearchWorld: Spatial Value-Grounded Imagination for UAV Object Search via World Models
National Key Laboratory of Digital Intelligent Modeling and Simulation, National University of Defense Technology（国防科技大学数字智能建模与仿真全国重点实验室）；College of Systems Engineering, National University of Defense Technology（国防科技大学系统工程学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；BNRist, Tsinghua University（清华大学北京国家研究中心）；Southeast University（东南大学）
Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games
Heriot-Watt University（赫瑞-瓦特大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Carnegie Mellon University（卡内基梅隆大学）
在视觉猜谜游戏中，猜谜者必须通过向预言者提问来识别场景中的目标对象。对于玩家来说，一个有效的策略是学习物体的概念表征，这些概念表征既有区分力又有足够的表达力，可以提出问题并正确猜测。然而，如Suglia et al.（2020）所示，现有模型无法学习真正的多模态表示，而是在训练和推理时依赖于场景中对象的黄金类别标签。当推理时的类别与训练时的类别相匹配时，这提供了不自然的性能优势，并且它导致模型在涉及域外对象类别的更真实的“zero-shot”场景中失败。为了克服这个问题，我们引入了一个基于正则化自动编码器的新型“想象”模块，该模块在推理时学习上下文感知和类别感知的潜在嵌入，而不依赖于类别标签。我们的想象力模块在CompGuessWhat？！中的游戏准确率超过了最先进的竞争对手8.26%。zero-shot方案（Suglia等人，2020），在GuessWhat？！中，它将Oracle和Guesser的准确率提高了2.08%和12.86%。基准，当推理时没有可用的黄金类别时。想象模块还可以增强对对象属性和属性的推理。
LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Driving
School of Mechanical Engineering, Southeast University（东南大学机械工程学院）；Department of Civil and Environmental Engineering, University of Michigan（密歇根大学土木与环境工程系）；Xheart Technology Co., Ltd.（芯驰科技有限公司）
