High-definition (HD) maps are crucial to autonomous driving, providing structured representations of road elements to support navigation and planning. However, existing query-based methods often employ random query initialization and depend on implicit temporal modeling, which lead to temporal inconsistencies and instabilities during the construction of a global map. To overcome these challenges, we introduce a novel end-to-end framework for consistent online HD vectorized map construction, which jointly performs map instance tracking and short-term prediction. First, we propose a Semantic-Aware Query Generator that initializes queries with spatially aligned semantic masks to capture scene-level context globally. Next, we design a History Rasterized Map Memory to store fine-grained instance-level maps for each tracked instance, enabling explicit historical priors. A History-Map Guidance Module then integrates rasterized map information into track queries, improving temporal continuity. Finally, we propose a Short-Term Future Guidance module to forecast the immediate motion of map instances based on the stored history trajectories. These predicted future locations serve as hints for tracked instances to further avoid implausible predictions and keep temporal consistency. Extensive experiments on the nuScenes and Argoverse2 datasets demonstrate that our proposed method outperforms state-of-the-art (SOTA) methods with good efficiency.
Huazhong University of Science & Technology（华中科技大学）；Horizon Robotics（地平线机器人）
High-definition (HD) map provides abundant and precise environmental information of the driving scene, serving as a fundamental and indispensable component for planning in autonomous driving system. We present MapTR, a structured end-to-end Transformer for efficient online vectorized HD map construction. We propose a unified permutation-equivalent modeling approach, i.e., modeling map element as a point set with a group of equivalent permutations, which accurately describes the shape of map element and stabilizes the learning process. We design a hierarchical query embedding scheme to flexibly…
MapDiffusion: Generative Diffusion for Vectorized Online HD Map Construction and Uncertainty Estimation in Autonomous Driving
Mercedes-Benz Research & Development North America（梅赛德斯-奔驰北美研发有限公司）；University of Stuttgart（斯图加特大学）；Institute for Artificial Intelligence（人工智能研究所）；University of California, San Diego（加州大学圣地亚哥分校）；University of Southampton（南安普顿大学）
MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction
Institute of Artificial Intelligence, Huazhong University of Science and Technology（华中科技大学人工智能研究院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；Horizon Robotics（地平线机器人）
NavMapFusion: Diffusion-based Fusion of Navigation Maps for Online Vectorized HD Map Construction
Mercedes-Benz Research & Development North America, USA（梅赛德斯-奔驰北美研究与开发）；University of Stuttgart, Germany（斯图加特大学）；University of California, San Diego, USA（加州大学圣地亚哥分校）；University of Southampton, United Kingdom（南安普顿大学）
MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expert
Lanzhou University（兰州大学）；Smart (Shanghai) Robotics Technology Co., Ltd.（智己（上海）机器人科技有限公司）；University of Science and Technology of China（中国科学技术大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Tsinghua University（清华大学）；Massachusetts Institute of Technology（麻省理工学院）；Li Auto（理想汽车）
Constructing HD semantic maps is a central component of autonomous driving. However, traditional pipelines require a vast amount of human efforts and resources in annotating and maintaining the semantics in the map, which limits its scalability. In this paper, we introduce the problem of HD semantic map learning, which dynamically constructs the local semantics based on onboard sensor observations. Meanwhile, we introduce a semantic map learning method, dubbed HDMapNet. HDMapNet encodes image features from surrounding cameras and/or point clouds from LiDAR, and predicts vectorized map elements…
OnlineBEV: Recurrent Temporal Fusion in Bird's Eye View Representations for Multi-Camera 3D Perception
Autonomous Driving Development Center, Hyundai Motors（韩世汽车自动驾驶开发中心）；Department of Electrical Engineering, Hanyang University（翰阳大学电气工程系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔国立大学人工智能跨学科项目）；Department of Electrical and Computer Engineering, Seoul National University（首尔国立大学电气与计算机工程系）
Online Vectorized HD Map Construction using Geometry
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；The Chinese University of Hong Kong（香港中文大学）；Tencent AI Lab（腾讯AI实验室）
Enhancing Online Road Network Perception and Reasoning with Standard Definition Maps
Bosch North America（博世北美）；Bosch Center for AI (BCAI)（博世人工智能中心）；Robert Bosch GmbH（罗伯特·博世有限公司）；Contextual Robotics Institute（情境机器人学研究所）；UC San Diego（加州大学圣迭戈分校）
Huazhong University of Science & Technology（华中科技大学）；Horizon Robotics（地平线机器人）
Motion prediction is highly relevant to the perception of dynamic objects and static map elements in the scenarios of autonomous driving. In this work, we propose PIP, the first end-to-end Transformer-based framework which jointly and interactively performs online mapping, object detection and motion prediction. PIP leverages map queries, agent queries and mode queries to encode the instance-wise information of map elements, agents and motion intentions, respectively. Based on the unified query representation, a differentiable multi-task interaction scheme is proposed to exploit the correlatio…
Gwangju Institute of Science and Technology（光州科学技术院）
In this paper, we propose the methods to handle temporal errors during multi-object tracking. Temporal error occurs when objects are occluded or noisy detections appear near the object. In those situations, tracking may fail and various errors like drift or ID-switching occur. It is hard to overcome temporal errors only by using motion and shape information. So, we propose the historical appearance matching method and joint-input siamese network which was trained by 2-step process. It can prevent tracking failures although objects are temporally occluded or last matching information is unrelia…
Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement…
Hunan University（湖南大学）；State Grid Hunan Electric Power Company Limited Economical Technical Research Institute（国网湖南省电力有限公司经济技术研究院）
Online multi-object tracking (MOT) is a longstanding task for computer vision and intelligent vehicle platform. At present, the main paradigm is tracking-by-detection, and the main difficulty of this paradigm is how to associate current candidate detections with historical tracklets. However, in the MOT scenarios, each historical tracklet is composed of an object sequence, while each candidate detection is just a flat image, which lacks temporal features of the object sequence. The feature difference between current candidate detections and historical tracklets makes the object association muc…
Kyushu University（九州大学）；Tianjin University of Technology（天津理工大学）；Alibaba Group（阿里巴巴集团）；Nanyang Technological University（南洋理工大学）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所（Amii））
Online updating of the object model via samples from historical frames is of great importance for accurate visual object tracking. Recent works mainly focus on constructing effective and efficient updating methods while neglecting the training samples for learning discriminative object models, which is also a key part of a learning problem. In this paper, we propose the DeepMix that takes historical samples' embeddings as input and generates augmented embeddings online, enhancing the state-of-the-art online learning methods for visual object tracking. More specifically, we first propose the on…
Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding
Wangxuan Institute of Computer Technology, Peking University（北京大学王宣计算机技术研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；Central Media Technology Institute, Huawei（华为中央媒体技术研究所）
Indiana University（印第安纳大学）；University of Maryland（马里兰大学）；Honda Research Institute, USA（美国本田研究所）
Most work on temporal action detection is formulated as an offline problem, in which the start and end times of actions are determined after the entire video is fully observed. However, important real-time applications including surveillance and driver assistance systems require identifying actions as soon as each video frame arrives, based only on current and historical observations. In this paper, we propose a novel framework, Temporal Recurrent Network (TRN), to model greater temporal context of a video frame by simultaneously performing online action detection and anticipation of the immed…
Huazhong University of Science and Technology（华中科技大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure. However, RNN suffers from non-parallelism and gradient vanishing, hence it is hard to be optimized. In this paper, we propose a new encoder-decoder framework based on Transformers, named OadTR, to tackle these problems. The encoder attached with a task token aims to capture the relationships and global interactions between historical observations. The decoder extracts auxiliary information by aggregating anticipated future clip representations. Therefore, Oa…
