In this paper, we focus on a less explored, but more realistic and complex problem of domain adaptation in LiDAR semantic segmentation. There is a significant drop in performance of an existing segmentation model when training (source domain) and testing (target domain) data originate from different LiDAR sensors. To overcome this shortcoming, we propose an unsupervised domain adaptation framework that leverages unlabeled target domain data for self-supervision, coupled with an unpaired mask transfer strategy to mitigate the impact of domain shifts. Furthermore, we introduce the gated adapter module with a small number of parameters into the network to account for target domain-specific information. Experiments adapting from both real-to-real and synthetic-to-real LiDAR semantic segmentation benchmarks demonstrate the significant improvement over prior arts.
Multi-Kernel Gated Decoder Adapters for Robust Multi-Task Thyroid Ultrasound under Cross-Center Shift
Department of Astronomy and Physics, University of British Columbia（不列颠哥伦比亚大学天文与物理系）；Department of Basic and Translational Research, BC Cancer Research Institute（不列颠哥伦比亚癌症研究所在基本与转化研究部）；Department of Radiology, University of British Columbia（不列颠哥伦比亚大学放射学系）；School of Biomedical Engineering, University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）
We present an imaging framework which converts three images from a gated camera into high-resolution depth maps with depth accuracy comparable to pulsed lidar measurements. Existing scanning lidar systems achieve low spatial resolution at large ranges due to mechanically-limited angular sampling rates, restricting scene understanding tasks to close-range clusters with dense sampling. Moreover, today's pulsed lidar scanners suffer from high cost, power consumption, large form-factors, and they fail in the presence of strong backscatter. We depart from point scanning and demonstrate that it is p…
Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation
East China University of Science and Technology（华东理工大学）；University of Sydney（悉尼大学）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学技术学院）
We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two-branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we further reduce the domain gap by inducing the model to learn a mapping between two domains using the domain shared and private features. Additionally, we introduce a new dataset (SemanticUSL\footnote…
Honda Research Institute, USA（美国本田研究院）；Carnegie Mellon University（卡内基梅隆大学）
The Tactical Driver Behavior modeling problem requires understanding of driver actions in complicated urban scenarios from a rich multi modal signals including video, LiDAR and CAN bus data streams. However, the majority of deep learning research is focused either on learning the vehicle/environment state (sensor fusion) or the driver policy (from temporal data), but not both. Learning both tasks end-to-end offers the richest distillation of knowledge, but presents challenges in formulation and successful training. In this work, we propose promising first steps in this direction. Inspired by t…
Gated cameras hold promise as an alternative to scanning LiDAR sensors with high-resolution 3D depth that is robust to back-scatter in fog, snow, and rain. Instead of sequentially scanning a scene and directly recording depth via the photon time-of-flight, as in pulsed LiDAR sensors, gated imagers encode depth in the relative intensity of a handful of gated slices, captured at megapixel resolution. Although existing methods have shown that it is possible to decode high-resolution depth from such measurements, these methods require synchronized and calibrated LiDAR to supervise the gated depth …
Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning
School of Information and Communication Engineering, BUPT（北京邮电大学信息与通信工程学院）；BUPT: Beijing University of Posts and Telecommunications（北京邮电大学）；Engineering Research Center for Information Network, Ministry of Education（教育部信息网络工程研究中心）
Today's state-of-the-art methods for 3D object detection are based on lidar, stereo, or monocular cameras. Lidar-based methods achieve the best accuracy, but have a large footprint, high cost, and mechanically-limited angular sampling rates, resulting in low spatial resolution at long ranges. Recent approaches based on low-cost monocular or stereo cameras promise to overcome these limitations but struggle in low-light or low-contrast regions as they rely on passive CMOS sensors. In this work, we propose a novel 3D object detection modality that exploits temporal illumination cues from a low-co…
Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation
School of Engineering, College of Engineering and Physical Sciences, University of Birmingham, Birmingham, UK（英国伯明翰大学工程学院）；School of Computer Science, University of Birmingham, Birmingham, UK（英国伯明翰大学计算机科学学院）；Department of Applied Health Sciences, University of Birmingham, Birmingham, UK（英国伯明翰大学应用健康科学系）
Gated imaging is an emerging sensor technology for self-driving cars that provides high-contrast images even under adverse weather influence. It has been shown that this technology can even generate high-fidelity dense depth maps with accuracy comparable to scanning LiDAR systems. In this work, we extend the recent Gated2Depth framework with aleatoric uncertainty providing an additional confidence measure for the depth estimates. This confidence can help to filter out uncertain estimations in regions without any illumination. Moreover, we show that training on dense depth maps generated by LiD…
R-FLoRA: Residual-Statistic-Gated Low-Rank Adaptation for Single-Image Face Morphing Attack Detection
SAFE Center, Department of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU)（安全中心，信息安全与通信技术系，挪威科学技术大学（NTNU））
ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Rhino（犀牛（公司））；The Chinese University of Hong Kong（香港中文大学）；OPPO Research Institute（OPPO研究院）；Zhejiang Lab（之江实验室）；ShanghaiTech University（上海科技大学）
Fudan University（复旦大学）；Meituan（美团）；Shanghai Collaborative Innovation Center on Intelligent Visual Computing（上海智能视觉计算协同创新中心）
Fusing LiDAR and camera information is essential for achieving accurate and reliable 3D object detection in autonomous driving systems. This is challenging due to the difficulty of combining multi-granularity geometric and semantic features from two drastically different modalities. Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images (referred to as seeds) into 3D space, and then incorporate 2D semantics via cross-modal interaction or fusion techniques. However, depth information is under-investigated in these approaches when …
GDBA-Net: A Gated Dual-Branch Adaptation of SAM2 for Robust Cross-Domain Nuclei Segmentation
University of Missouri（密苏里大学）；Lander University（兰德大学）；Sagarmatha Engineering College, Tribhuvan University（萨加玛塔工程学院，特里布万大学）；Morsani College of Medicine, University of South Florida（莫尔萨尼医学院，南佛罗里达大学）
Hikvision Research Institute（海康威视研究院）；Zhejiang University（浙江大学）
Point clouds can be represented in many forms (views), typically, point-based sets, voxel-based cells or range-based images(i.e., panoramic view). The point-based view is geometrically accurate, but it is disordered, which makes it difficult to find local neighbors efficiently. The voxel-based view is regular, but sparse, and computation grows cubically when voxel resolution increases. The range-based view is regular and generally dense, however spherical projection makes physical dimensions distorted. Both voxel- and range-based views suffer from quantization loss, especially for voxels when …
Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT
University Health Network（大学健康网络）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；Islamic Azad University（伊斯兰阿扎德大学）
When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation
University of Delaware（特拉华大学）；University of South Florida（南佛罗里达大学）
Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation
College of Land Science and Technology, China Agricultural University（中国农业大学土地科学与技术学院）；Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture and Rural Affairs（农业农村部农业灾害遥感重点实验室）；Faculty of Geosciences and Engineering, Southwest Jiaotong University（西南交通大学地质科学与工程学院）；School of Artificial Intelligence, Sun Yat-Sen University（中山大学人工智能学院）；Henan Polytechnic University（河南理工大学）；Key Laboratory of Spatio-Temporal Information and Ecological Restoration of Mines, Ministry of Natural Resources of the People’s Republic of China（矿产资源时空信息与生态修复重点实验室）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；National Supercomputing Center in Shenzhen, Shenzhen, China（深圳国家超算中心）；Ministry of Education Key Laboratory for Earth System Modeling and the Department of Earth System Science, Tsinghua University（地球系统模拟教育部重点实验室和清华大学地球系统科学系）
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院人形机器人与智能系统研究所）；Hangzhou SurImage Technology Company Ltd.（杭州 SurImage 科技有限公司）；Hangzhou KrVision Technology Company Ltd.（杭州 KrVision 科技有限公司）
As the scene information, including objectness and scene type, are important for people with visual impairment, in this work we present a multi-task efficient perception system for the scene parsing and recognition tasks. Building on the compact ResNet backbone, our designed network architecture has two paths with shared parameters. In the structure, the semantic segmentation path integrates fast attention, with the aim of harvesting long-range contextual information in an efficient manner. Simultaneously, the scene recognition path attains the scene type inference by passing the semantic feat…
