Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper proposes a novel and general framework called PriorLane, which is used to enhance the segmentation performance of the fully vision transformer by introducing the low-cost local prior knowledge. Specifically, PriorLane utilizes an encoder-only transformer to fuse the feature extracted by a pre-trained segmentation model with prior knowledge embeddings. Note that a Knowledge Embedding Alignment (KEA) module is adapted to enhance the fusion performance by aligning the knowledge embedding. Extensive experiments on our Zjlab dataset show that PriorLane outperforms SOTA lane detection methods by a 2.82% mIoU when prior knowledge is employed, and the code will be released at: https://github.com/vincentqqb/PriorLane.
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）
We present Laneformer, a conceptually simple yet powerful transformer-based architecture tailored for lane detection that is a long-standing research topic for visual perception in autonomous driving. The dominant paradigms rely on purely CNN-based architectures which often fail in incorporating relations of long-range lane points and global contexts induced by surrounding objects (e.g., pedestrians, vehicles). Inspired by recent advances of the transformer encoder-decoder architecture in various vision tasks, we move forwards to design a new end-to-end Laneformer architecture that revolutioni…
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）
Methods for 3D lane detection have been recently proposed to address the issue of inaccurate lane layouts in many autonomous driving scenarios (uphill/downhill, bump, etc.). Previous work struggled in complex cases due to their simple designs of the spatial transformation between front view and bird's eye view (BEV) and the lack of a realistic dataset. Towards these issues, we present PersFormer: an end-to-end monocular 3D lane detector with a novel Transformer-based spatial feature transformation module. Our model generates BEV features by attending to related front-view local regions with ca…
NullMax；School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）
3D lane detection is an integral part of autonomous driving systems. Previous CNN and Transformer-based methods usually first generate a bird's-eye-view (BEV) feature map from the front view image, and then use a sub-network with BEV feature map as input to predict 3D lanes. Such approaches require an explicit view transformation between BEV and front view, which itself is still a challenging problem. In this paper, we propose CurveFormer, a single-stage Transformer-based method that directly calculates 3D lane parameters and can circumvent the difficult view transformation step. Specifically,…
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
With the fast development of autonomous driving technologies, there is an increasing demand for high-definition (HD) maps, which provide reliable and robust prior information about the static part of the traffic environments. As one of the important elements in HD maps, road lane centerline is critical for downstream tasks, such as prediction and planning. Manually annotating centerlines for road lanes in HD maps is labor-intensive, expensive and inefficient, severely restricting the wide applications of autonomous driving systems. Previous work seldom explores the lane centerline detection pr…
National Central University（国立中央大学）；Tokyo Metropolitan University（东京都立大学）；Auburn University（奥本大学）
Reasoning about vehicle path prediction is an essential and challenging problem for the safe operation of autonomous driving systems. There exist many research works for path prediction. However, most of them do not use lane information and are not based on the Transformer architecture. By utilizing different types of data collected from sensors equipped on the self-driving vehicles, we propose a path prediction system named Multi-modal Transformer Path Prediction (MTPP) that aims to predict long-term future trajectory of target agents. To achieve more accurate path prediction, the Transformer…
LMT-Net: Lane Model Transformer Network for Automated HD Mapping from Sparse Vehicle Observations
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Mercedes-Benz Research & Development North America（梅赛德斯-奔驰北美研发中心）；University of Stuttgart（斯图加特大学）；University of Southampton（南安普顿大学）
Lane detection is one of the core functions in autonomous driving and has aroused widespread attention recently. The networks to segment lane instances, especially with bad appearance, must be able to explore lane distribution properties. Most existing methods tend to resort to CNN-based techniques. A few have a try on incorporating the recent adorable, the seq2seq Transformer \cite{transformer}. However, their innate drawbacks of weak global information collection ability and exorbitant computation overhead prohibit a wide range of the further applications. In this work, we propose Atrous Tra…
Lane detection is a fundamental task in autonomous driving, and has achieved great progress as deep learning emerges. Previous anchor-based methods often design dense anchors, which highly depend on the training dataset and remain fixed during inference. We analyze that dense anchors are not necessary for lane detection, and propose a transformer-based lane detection framework based on a sparse anchor mechanism. To this end, we generate sparse anchors with position-aware lane queries and angle queries instead of traditional explicit anchors. We adopt Horizontal Perceptual Attention (HPA) to ag…
RATopo: Improving Lane Topology Reasoning via Redundancy Assignment
School of Artificial Intelligence Beihang University Beijing China（人工智能学院 北航 北京 中国）；Zhongguancun Academy Beijing China（中关村学院 北京 中国）；Hangzhou International Innovation Institute Beihang University Hangzhou China（杭州国际创新研究院 北航 杭州 中国）；Beihang University（北航）；Zhongguancun Academy（中关村学院）；University of Macau（澳门大学）
Reusing Attention for One-stage Lane Topology Understanding
Institute for AI Industry Research, Tsinghua University（人工智能产业研究院，清华大学）；Bosch Corporate Research, China（博世企业研究，中国）；Institute for Interdisciplinary Information Sciences, Tsinghua University（交叉信息科学研究院，清华大学）；Department of Computer Science, ETH Zürich（计算机科学系，苏黎世联邦理工学院）；State Key Lab of Intelligent Transportation System, Beihang University（智能交通系统国家重点实验室，北航）
A Concise Survey on Lane Topology Reasoning for HD Mapping
NavInfo Co., Ltd.（四维图新有限公司）；Autohome Inc.（汽车之家）；Baidu Inc.（百度公司）；Xidian University of Technology（西安工业大学）；Hefei University of Technology（合肥工业大学）；Renmin University of China（中国人民大学）
Accurate detection of lane and road markings is a task of great importance for intelligent vehicles. In existing approaches, the detection accuracy often degrades with the increasing distance. This is due to the fact that distant lane and road markings occupy a small number of pixels in the image, and scales of lane and road markings are inconsistent at various distances and perspectives. The Inverse Perspective Mapping (IPM) can be used to eliminate the perspective distortion, but the inherent interpolation can lead to artifacts especially around distant lane and road markings and thus has a …
SparseLaneSTP: Leveraging Spatio-Temporal Priors with Sparse Transformers for 3D Lane Detection
Bosch Mobility Solutions, Robert Bosch GmbH（博世移动解决方案，罗伯特·博世有限公司）；Institute of Neuro- and Bioinformatics, University of Lübeck（神经与生物医学研究所，吕贝克大学）；Institute for Signal Processing and System Theory, University of Stuttgart（信号处理与系统理论研究所，斯图加特大学）
Self-supervised learning (SSL) is an emerging technique that has been successfully employed to train convolutional neural networks (CNNs) and graph neural networks (GNNs) for more transferable, generalizable, and robust representation learning. However its potential in motion forecasting for autonomous driving has rarely been explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into motion forecasting. We first propose to investigate four novel self-supervised learning tasks for motion forecasting with theoretical rationale and qu…
ARETE: Attention-based Rasterized Encoding for Topology Estimation using HSV-transformed Crowdsourced Vehicle Fleet Data
Mercedes-Benz AG, Research \& Development, Sindelfingen, Germany Institute for Intelligent Systems, Esslingen University of Applied Sciences, Esslingen, Germany
Lane Departure Accident Prevention in Foggy Conditions: A Prior-Guided Dynamic Feature Fusion Transformer Framework for Real-Time Lane Detection
Guangdong Key Laboratory of Intelligent Transportation System, School of Intelligent Systems Engineering, Sun Yat-sen University（广东省智能交通系统重点实验室，智能系统工程学院，中山大学）；School of Civil Engineering and Transportation, South China University of Technology（土木工程与交通学院，华南理工大学）；College of Automotive Studies, Tongji University（汽车研究学院，同济大学）；College of Transportation Engineering, Tongji University（交通运输工程学院，同济大学）；Department of Civil and Environmental Engineering, University of Alberta（土木与环境工程系，阿尔伯塔大学）；State Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University（汽车底盘集成与生物力学国家重点实验室，吉林大学）
Xi’an Jiaotong University（西安交通大学）；Capital Normal University（首都师范大学）；Shenzhen Forward Innovation Digital Technology Co. Ltd（深圳市前沿创新数字技术有限公司）
Lane detection, the process of identifying lane markings as approximated curves, is widely used for lane departure warning and adaptive cruise control in autonomous vehicles. The popular pipeline that solves it in two steps -- feature extraction plus post-processing, while useful, is too inefficient and flawed in learning the global context and lanes' long and thin structures. To tackle these issues, we propose an end-to-end method that directly outputs parameters of a lane shape model, using a network built with a transformer to learn richer structures and context. The lane shape model is for…
PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer
Department of Electrical Engineering and Information Technology, Intelligent Vehicles Lab (IVL), Munich University of Applied Science（电气工程与信息科技系，智能车辆实验室（IVL），慕尼黑应用科学大学）
LaneTCA: Enhancing Video Lane Detection with Temporal Context Aggregation
National Engineering Laboratory for Brain-Inspired Intelligence Technology and Application（类脑智能技术与应用国家工程实验室）；University of Science and Technology of China（中国科学技术大学）
