False negatives (FN) in 3D object detection, {\em e.g.}, missing predictions of pedestrians, vehicles, or other obstacles, can lead to potentially dangerous situations in autonomous driving. While being fatal, this issue is understudied in many current 3D detection methods. In this work, we propose Hard Instance Probing (HIP), a general pipeline that identifies \textit{FN} in a multi-stage manner and guides the models to focus on excavating difficult instances. For 3D object detection, we instantiate this method as FocalFormer3D, a simple yet effective detector that excels at excavating difficult objects and improving prediction recall. FocalFormer3D features a multi-stage query generation to discover hard objects and a box-level transformer decoder to efficiently distinguish objects from massive object candidates. Experimental results on the nuScenes and Waymo datasets validate the superior performance of FocalFormer3D. The advantage leads to strong performance on both detection and tracking, in both LiDAR and multi-modal settings. Notably, FocalFormer3D achieves a 70.5 mAP and 73.9 NDS on nuScenes detection benchmark, while the nuScenes tracking benchmark shows 72.1 AMOTA, both ranking 1st place on the nuScenes LiDAR leaderboard. Our code is available at \url{https://github.com/NVlabs/FocalFormer3D}.
Center Focusing Network for Real-Time LiDAR Panoptic Segmentation
Beijing Municipal Key Lab of Multimedia and Intelligent Software Technology(北京市多媒体与智能软件技术重点实验室);Beijing Institute of Artificial Intelligence(北京人工智能研究院);Beijing University of Technology(北京工业大学);Mogo Auto Intelligence and Telemetics Information Technology Co. Ltd.(摩戈汽车智能与车载信息技术有限公司)
Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Tencent Wechat AI(腾讯微信AI);School of Information and Communication Engineering, Beijing University of Posts and Telecommunications(北京邮电大学信息与通信工程学院);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院)
Online progressive instance-balanced sampling for weakly supervised object detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院)
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)(德国航空航天中心机器人与机电一体化研究所);Department of Computer Science, Technical University of Munich (TUM)(慕尼黑工业大学计算机科学系)
Multiple Instance Learning Framework with Masked Hard Instance Mining for Gigapixel Histopathology Image Analysis
Hong Kong University of Science and Technology(香港科学与技术大学);Hefei University of Technology(合肥工业大学);Nanjing University of Posts and Telecommunications(南京邮电大学)
Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation
University of Science and Technology of China (USTC)(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);Microsoft Research Asia(微软亚洲研究院)
Rethinking Multiple Instance Learning: Developing an Instance-Level Classifier via Weakly-Supervised Self-Training
Digital Medical Research Center, School of Basic Medical Science, Fudan University(复旦大学基础医学院数字医学研究中心);Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention(上海市医学影像计算与计算机辅助介入重点实验室)
SSR-HEF: Crowd Counting with Multi-Scale Semantic Refining and Hard Example Focusing
Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences(中国科学院合肥物质科学研究院智能机械研究所);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学);University of Sydney(悉尼大学);Zhejiang Sci-Tech University(浙江理工大学)
基于密度图的人群计数通常被视为回归任务。深度学习用于学习图像内容与人群密度分布之间的映射。虽然取得了很大的成功,但一些远离摄像头的行人很难被发现。而且硬例子的数量往往更大。现有的基于简单欧氏距离算法的算法不加区分地对难样本和易样本进行优化,使得难样本的密度通常被错误地预测为较低甚至为零,从而导致较大的计数误差。为了解决这个问题,我们首先提出了Hard Example Focusing(HEF)算法,用于人群计数的回归任务。HEF算法通过衰减简单样本的贡献,使模型快速聚焦于困难样本,从而提高估计错误的困难样本的重要性。此外,人群场景中的比例变化很大,并且比例标注是劳动密集型的并且昂贵。通过提出多尺度语义提炼(SSR)策略,模型的底层可以突破深度学习的局限性,捕捉不同尺度的语义特征,以充分应对尺度变化。我们在六个基准数据集上进行了大量的实验来验证所提出的方法。结果表明,我们所提出的方法比国家的最先进的方法的优越性。此外,我们设计的模型更小,更快。
Interventional Multi-Instance Learning with Deconfounded Instance-Level Prediction
Shanghai Jiao Tong University(上海交通大学);Renmin University of China(中国人民大学);Beijing Key Laboratory of Big Data Management and Analysis Methods(北京大数据管理与分析方法重点实验室);JD Explore Academy(京东探索研究院)
Dictionary-based Pathology Mining with Hard-instance-assisted Classifier Debiasing for Genetic Biomarker Prediction from WSIs
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University(华东师范大学上海市多维度信息处理重点实验室);Department of General Surgery, Tongji Hospital, Tongji University School of Medicine(同济大学医学院附属同济医院普外科)