Dream to Drive with Predictive Individual World Model
School of Automation and Electrical Engineering, University of Science and Technology Beijing(北京科技大学自动化与电气工程学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
It is still a challenging topic to make reactive driving behaviors in complex urban environments as road users' intentions are unknown. Model-based reinforcement learning (MBRL) offers great potential to learn a reactive policy by constructing a world model that can provide informative states and imagination training. However, a critical limitation in relevant research lies in the scene-level reconstruction representation learning, which may overlook key interactive vehicles and hardly model the interactive features among vehicles and their long-term intentions. Therefore, this paper presents a novel MBRL method with a predictive individual world model (PIWM) for autonomous driving. PIWM describes the driving environment from an individual-level perspective and captures vehicles' interactive relations and their intentions via trajectory prediction task. Meanwhile, a behavior policy is learned jointly with PIWM. It is trained in PIWM's imagination and effectively navigates in the urban driving scenes leveraging intention-aware latent states. The proposed method is trained and evaluated on simulation environments built upon real-world challenging interactive scenarios. Compared with popular model-free and state-of-the-art model-based reinforcement learning methods, experimental results show that the proposed method achieves the best performance in terms of safety and efficiency.
DreamReg: Belief-Driven World Model for 2D-3D Ultrasound Registration
T Stone Robotics Institute, The Chinese University of Hong Kong(香港中文大学T Stone机器人研究所);Multi-scale Medical Robotics Center(多尺度医疗机器人中心);Perelman School of Medicine, University of Pennsylvania(宾夕法尼亚大学佩雷尔曼医学院)
Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation
Center for Mind/Brain Sciences(心智/脑科学中心);University of Trento(特伦托大学);Institute of Formal and Applied Linguistics(形式与应用语言学研究所);Charles University of Prague(布拉格查理大学)
DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Technical University of Munich(慕尼黑技术大学)
DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Driving
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);Shanghai Jiao Tong University(上海交通大学);East China Normal University(华东师范大学);Technical University of Munich(慕尼黑工业大学)
DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation
Beijing Institute of Technology(北京理工大学);ETH Zurich(苏黎世联邦理工学院);Zhejiang University(浙江大学);Yangtze Delta Region Academy of Beijing Institute of Technology(北京理工大学长三角研究院)
Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination
Autonomous Intelligent Systems, Computer Science Institute VI - Intelligent Systems(自主智能系统,计算机科学研究所VI - 智能系统);Robotics, Center for Robotics(机器人学,机器人中心);the Lamarr Institute for Machine Learning(拉马尔机器学习研究所);Artificial Intelligence, University of Bonn, Germany(人工智能,波恩大学,德国)
Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation
National Key Laboratory of Human Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University(西安交通大学人机混合增强智能全国重点实验室);School of Automation and Intelligent Sensing, Shanghai Jiao Tong University(上海交通大学自动化与智能感知学院)
Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA
Department of Computer Science, SCMNS School, Morgan State University, Baltimore, Maryland 21251, USA;Electrical \& Computer Engineering Dept., School of Engineering, Morgan State University, Baltimore, Maryland 21251, USA
LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Driving
School of Mechanical Engineering, Southeast University(东南大学机械工程学院);Department of Civil and Environmental Engineering, University of Michigan(密歇根大学土木与环境工程系);Xheart Technology Co., Ltd.(芯驰科技有限公司)
Rethinking Driving World Model as Synthetic Data Generator for Perception Tasks
Peking University(北京大学);Xiaomi EV(小米电动车);Huazhong University of Science and Technology(华中科技大学);Beijing Key Laboratory of Data Intelligence and Security (Peking University)(北京数据智能与安全重点实验室(北京大学));Zhongguancun Academy(中关村学院)
Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
Peking University(北京大学);The Hong Kong University of Science and Technology(香港科技大学);Nanjing University(南京大学);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室)