3D-LaneNet+ is a camera-based DNN method for anchor free 3D lane detection which is able to detect 3d lanes of any arbitrary topology such as splits, merges, as well as short and perpendicular lanes. We follow recently proposed 3D-LaneNet, and extend it to enable the detection of these previously unsupported lane topologies. Our output representation is an anchor free, semi-local tile representation that breaks down lanes into simple lane segments whose parameters can be learnt. In addition we learn, per lane instance, feature embedding that reasons for the global connectivity of locally detected segments to form full 3d lanes. This combination allows 3D-LaneNet+ to avoid using lane anchors, non-maximum suppression, and lane model fitting as in the original 3D-LaneNet. We demonstrate the efficacy of 3D-LaneNet+ using both synthetic and real world data. Results show significant improvement relative to the original 3D-LaneNet that can be attributed to better generalization to complex lane topologies, curvatures and surface geometries.
We present a generalized and scalable method, called Gen-LaneNet, to detect 3D lanes from a single image. The method, inspired by the latest state-of-the-art 3D-LaneNet, is a unified framework solving image encoding, spatial transform of features and 3D lane prediction in a single network. However, we propose unique designs for Gen-LaneNet in two folds. First, we introduce a new geometry-guided lane anchor representation in a new coordinate frame and apply a specific geometric transformation to directly calculate real 3D lane points from the network output. We demonstrate that aligning the lan…
Duke University（杜克大学）；Horizon Robotics, Inc.（地平线机器人公司）
Lane detection is to detect lanes on the road and provide the accurate location and shape of each lane. It severs as one of the key techniques to enable modern assisted and autonomous driving systems. However, several unique properties of lanes challenge the detection methods. The lack of distinctive features makes lane detection algorithms tend to be confused by other objects with similar local appearance. Moreover, the inconsistent number of lanes on a road as well as diverse lane line patterns, e.g. solid, broken, single, double, merging, and splitting lines further hamper the performance. …
German Aerospace Center (DLR)（德国航空航天中心（DLR））；Technical University of Munich（慕尼黑工业大学）
The knowledge about the placement and appearance of lane markings is a prerequisite for the creation of maps with high precision, necessary for autonomous driving, infrastructure monitoring, lane-wise traffic management, and urban planning. Lane markings are one of the important components of such maps. Lane markings convey the rules of roads to drivers. While these rules are learned by humans, an autonomous driving vehicle should be taught to learn them to localize itself. Therefore, accurate and reliable lane marking semantic segmentation in the imagery of roads and highways is needed to ach…
We introduce a network that directly predicts the 3D layout of lanes in a road scene from a single image. This work marks a first attempt to address this task with on-board sensing without assuming a known constant lane width or relying on pre-mapped environments. Our network architecture, 3D-LaneNet, applies two new concepts: intra-network inverse-perspective mapping (IPM) and anchor-based lane representation. The intra-network IPM projection facilitates a dual-representation information flow in both regular image-view and top-view. An anchor-per-column output representation enables our end-t…
Compared to 2D lanes, real 3D lane data is difficult to collect accurately. In this paper, we propose a novel method for training 3D lanes with only 2D lane labels, called weakly supervised 3D lane detection WS-3D-Lane. By assumptions of constant lane width and equal height on adjacent lanes, we indirectly supervise 3D lane heights in the training. To overcome the problem of the dynamic change of the camera pitch during data collection, a camera pitch self-calibration method is proposed. In anchor representation, we propose a double-layer anchor with a improved non-maximum suppression (NMS) me…
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；TuSimple（图森未来）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Monocular 3D lane detection is a challenging task due to its lack of depth information. A popular solution is to first transform the front-viewed (FV) images or features into the bird-eye-view (BEV) space with inverse perspective mapping (IPM) and detect lanes from BEV features. However, the reliance of IPM on flat ground assumption and loss of context information make it inaccurate to restore 3D information from BEV representations. An attempt has been made to get rid of BEV and predict 3D lanes from FV representations directly, while it still underperforms other BEV-based methods given its l…
Lane detection is a fundamental task in autonomous driving, and has achieved great progress as deep learning emerges. Previous anchor-based methods often design dense anchors, which highly depend on the training dataset and remain fixed during inference. We analyze that dense anchors are not necessary for lane detection, and propose a transformer-based lane detection framework based on a sparse anchor mechanism. To this end, we generate sparse anchors with position-aware lane queries and angle queries instead of traditional explicit anchors. We adopt Horizontal Perceptual Attention (HPA) to ag…
Anchor3DLane++: 3D Lane Detection via Sample-Adaptive Sparse 3D Anchor Regression
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；TuSimple（图森未来）；The Chinese University of Hong Kong（香港中文大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）；Zhejiang–Singapore Innovation and AI Joint Research Lab（浙江-新加坡创新与人工智能联合实验室）
Modern methods mainly regard lane detection as a problem of pixel-wise segmentation, which is struggling to address the problems of efficiency and challenging scenarios like severe occlusions and extreme lighting conditions. Inspired by human perception, the recognition of lanes under severe occlusions and extreme lighting conditions is mainly based on contextual and global information. Motivated by this observation, we propose a novel, simple, yet effective formulation aiming at ultra fast speed and the problem of challenging scenarios. Specifically, we treat the process of lane detection as …
Recently, lane detection has made great progress with the rapid development of deep neural networks and autonomous driving. However, there exist three mainly problems including characterizing lanes, modeling the structural relationship between scenes and lanes, and supporting more attributes (e.g., instance and type) of lanes. In this paper, we propose a novel structure guided framework to solve these problems simultaneously. In the framework, we first introduce a new lane representation to characterize each instance. Then a topdown vanishing point guided anchoring mechanism is proposed to pro…
Universidade Federal do Espírito Santo (UFES)（圣埃斯皮里图联邦大学）；Instituto Federal do Espírito Santo (IFES)（圣埃斯皮里图联邦学院）
Modern lane detection methods have achieved remarkable performances in complex real-world scenarios, but many have issues maintaining real-time efficiency, which is important for autonomous vehicles. In this work, we propose LaneATT: an anchor-based deep lane detection model, which, akin to other generic deep object detectors, uses the anchors for the feature pooling step. Since lanes follow a regular pattern and are highly correlated, we hypothesize that in some cases global information may be crucial to infer their positions, especially in conditions such as occlusion, missing lane markers, …
TIFAC-CORE in Automotive Infotronics（汽车信息电子技术TIFAC核心中心）；Vellore Institute of Technology（韦洛尔理工学院）
In a self-driving car, objection detection, object classification, lane detection and object tracking are considered to be the crucial modules. In recent times, using the real time video one wants to narrate the scene captured by the camera fitted in our vehicle. To effectively implement this task, deep learning techniques and automatic video annotation tools are widely used. In the present paper, we compare the various techniques that are available for each module and choose the best algorithm among them by using appropriate metrics. For object detection, YOLO and Retinanet-50 are considered …
DALNet: A Rail Detection Network Based on Dynamic Anchor Line
School of Control Science and Engineering, Dalian University of Technology（大连理工大学控制科学与工程学院）；Dalian Rail Transmit Intelligent Control and Intelligent Operation Technology Innovation Center（大连轨道交通智能控制与智能运维技术创新中心）；Dalian Seasky Automation Co., Ltd（大连海空自动化有限公司）
Peking University（北京大学）；SenseTime Research（商汤科技研究院）；University of Science and Technology of China（中国科学技术大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Lane detection is a challenging task that requires predicting complex topology shapes of lane lines and distinguishing different types of lanes simultaneously. Earlier works follow a top-down roadmap to regress predefined anchors into various shapes of lane lines, which lacks enough flexibility to fit complex shapes of lanes due to the fixed anchor shapes. Lately, some works propose to formulate lane detection as a keypoint estimation problem to describe the shapes of lane lines more flexibly and gradually group adjacent keypoints belonging to the same lane line in a point-by-point manner, whi…
We present ONCE-3DLanes, a real-world autonomous driving dataset with lane layout annotation in 3D space. Conventional 2D lane detection from a monocular image yields poor performance of following planning and control tasks in autonomous driving due to the case of uneven road. Predicting the 3D lane layout is thus necessary and enables effective and safe driving. However, existing 3D lane detection datasets are either unpublished or synthesized from a simulated environment, severely hampering the development of this field. In this paper, we take steps towards addressing these issues. By exploi…
NullMax；School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）
3D lane detection is an integral part of autonomous driving systems. Previous CNN and Transformer-based methods usually first generate a bird's-eye-view (BEV) feature map from the front view image, and then use a sub-network with BEV feature map as input to predict 3D lanes. Such approaches require an explicit view transformation between BEV and front view, which itself is still a challenging problem. In this paper, we propose CurveFormer, a single-stage Transformer-based method that directly calculates 3D lane parameters and can circumvent the difficult view transformation step. Specifically,…
Rethinking Lanes and Points in Complex Scenarios for Monocular 3D Lane Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；PhiGent Robotics（PhiGent机器人公司）；UCAS（中国科学院大学）；Luoyang Institute for Robot and Intelligent Equipment（洛阳机器人与智能装备研究院）
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）
Methods for 3D lane detection have been recently proposed to address the issue of inaccurate lane layouts in many autonomous driving scenarios (uphill/downhill, bump, etc.). Previous work struggled in complex cases due to their simple designs of the spatial transformation between front view and bird's eye view (BEV) and the lack of a realistic dataset. Towards these issues, we present PersFormer: an end-to-end monocular 3D lane detector with a novel Transformer-based spatial feature transformation module. Our model generates BEV features by attending to related front-view local regions with ca…
Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；SenseTime Research（商汤科技研究院）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；State Key Laboratory of Complex & Critical Software Environment, Beihang University（北京航空航天大学复杂与关键软件环境国家重点实验室）
Rethinking Efficient Lane Detection via Curve Modeling
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；City University of Hong Kong（香港城市大学）；SenseTime Research（商汤科技研究院）；Qing Yuan Research Institute, Shanghai Jiao Tong University（上海交通大学清源研究院）
