Self-supervised depth estimators have recently shown results comparable to the supervised methods on the challenging single image depth estimation (SIDE) task, by exploiting the geometrical relations between target and reference views in the training data. However, previous methods usually learn forward or backward image synthesis, but not depth estimation, as they cannot effectively neglect occlusions between the target and the reference images. Previous works rely on rigid photometric assumptions or the SIDE network to infer depth and occlusions, resulting in limited performance. On the other hand, we propose a method to "Forget About the LiDAR" (FAL), for the training of depth estimators, with Mirrored Exponential Disparity (MED) probability volumes, from which we obtain geometrically inspired occlusion maps with our novel Mirrored Occlusion Module (MOM). Our MOM does not impose a burden on our FAL-net. Contrary to the previous methods that learn SIDE from stereo pairs by regressing disparity in the linear space, our FAL-net regresses disparity by binning it into the exponential space, which allows for better detection of distant and nearby objects. We define a two-step training strategy for our FAL-net: It is first trained for view synthesis and then fine-tuned for depth estimation with our MOM. Our FAL-net is remarkably light-weight and outperforms the previous state-of-the-art methods with 8x fewer parameters and 3x faster inference speeds on the challenging KITTI dataset. We present extensive experimental results on the KITTI, CityScapes, and Make3D datasets to verify our method's effectiveness. To the authors' best knowledge, the presented method performs the best among all the previous self-supervised methods until now.
Neural networks are susceptible to catastrophic forgetting. They fail to preserve previously acquired knowledge when adapting to new tasks. Inspired by human associative memory system, we propose a brain-like approach that imitates the associative learning process to achieve continual learning. We design a heuristics mechanism to potentiatively stimulate the model, which guides the model to recall the historical episodes based on the current circumstance and obtained association experience. Besides, a distillation measure is added to depressively alter the efficacy of synaptic transmission, wh…
Università degli Studi di Milano - Bicocca（米兰比可卡大学）；Universidad de Alcalá（阿尔卡拉大学）；Albert-Ludwigs-Universität Freiburg（阿尔伯特-路德维希-弗莱堡大学）
In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i.e. CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07deg median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17deg. To the best of ou…
LASTIG, Univ. Gustave Eiffel, IGN-ENSG（古斯塔夫·埃菲尔大学LASTIG实验室，法国国家地理与林业信息学院）；INRAE, UMR 1202 BIOGECO, Université de Bordeaux（法国国家农业食品与环境研究院，波尔多大学生物地球化学与生态学联合研究中心）；EA G&E Bordeaux INP, Université Bordeaux Montaigne（波尔多蒙田大学，波尔多国立高等电子、计算机、电信、数学与机械学院环境与地理空间工程研究团队）
We propose a new deep learning-based method for estimating the occupancy of vegetation strata from airborne 3D LiDAR point clouds. Our model predicts rasterized occupancy maps for three vegetation strata corresponding to lower, medium, and higher cover. Our weakly-supervised training scheme allows our network to only be supervised with vegetation occupancy values aggregated over cylindrical plots containing thousands of points. Such ground truth is easier to produce than pixel-wise or point-wise annotations. Our method outperforms handcrafted and deep learning baselines in terms of precision b…
This paper proposes RIU-Net (for Range-Image U-Net), the adaptation of a popular semantic segmentation network for the semantic segmentation of a 3D LiDAR point cloud. The point cloud is turned into a 2D range-image by exploiting the topology of the sensor. This image is then used as input to a U-net. This architecture has already proved its efficiency for the task of semantic segmentation of medical images. We demonstrate how it can also be used for the accurate semantic segmentation of a 3D LiDAR point cloud and how it represents a valid bridge between image processing and 3D point cloud pro…
Shanghai Jiao Tong University（上海交通大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Shanghai AI Laboratory（上海人工智能实验室）
Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems. However, a major challenge lies in precise depth estimation due to the lack of direct 3D measurements in the input. Many previous methods attempt to improve depth estimation through network designs, e.g., deformable layers and larger receptive fields. This work proposes an orthogonal direction, improving the camera-only 3D detection by introducing multi-agent collaborations. Our proposed collaborative camera-only 3D detection (CoCa3D…
This paper is about extremely robust and lightweight localisation using LiDAR point clouds based on instance segmentation and graph matching. We model 3D point clouds as fully-connected graphs of semantically identified components where each vertex corresponds to an object instance and encodes its shape. Optimal vertex association across graphs allows for full 6-Degree-of-Freedom (DoF) pose estimation and place recognition by measuring similarity. This representation is very concise, condensing the size of maps by a factor of 25 against the state-of-the-art, requiring only 3kB to represent a 1…
EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media
Center for Modeling, Simulation, & Imaging in Medicine（建模、仿真与成像医学中心）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；University at Albany（阿尔巴尼大学）；IBM T.J. Watson Research Center（IBM 沃森研究中心）
TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation
Zhejiang University（浙江大学）；Shanghai AI Laboratory（上海人工智能实验室）；Fullong Inc.（福隆公司）；School of Software Technology, Zhejiang University（浙江大学软件学院）；The Chinese University of Hong Kong（香港中文大学）；ShanghaiTech University（上海科技大学）
Block Expanded DINORET: Adapting Natural Domain Foundation Models for Retinal Imaging Without Catastrophic Forgetting
Stadtspital Zürich（苏黎世市立医院）；Spross Research Institute（斯普罗斯研究所）；Gutblick Research（古特布里克研究公司）；Balgrist University Hospital（巴尔格里斯特大学医院）；Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔德眼科医院国民保健服务基金会信托）；Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；University College London（伦敦大学学院）；National University of Singapore（新加坡国立大学）；Singapore Eye Research Institute, Singapore National Eye Centre（新加坡国家眼科中心新加坡眼科研究所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；iScreen 2 Prevent LLC（iScreen 2 Prevent有限责任公司）；Bascom Palmer Eye Institute, University of Miami（迈阿密大学巴斯康帕默眼科研究所）；University of Heidelberg（海德堡大学）；Semmelweis University（塞梅尔魏斯大学）
InvCoSS: Inversion-driven Continual Self-supervised Learning in Medical Multi-modal Image Pre-training
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Brain-Computer Interface & Brain-Inspired Intelligence Key Laboratory of Sichuan Province（四川省脑机接口与脑启发智能重点实验室）
Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算机与通讯学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡资讯研究院，科技研究局（A*STAR））；Department of Automation, Tsinghua University（清华大学自动化系）
