DI-MaskDINO：一种联合目标检测与实例分割模型
DI-MaskDINO: A Joint Object Detection and Instance Segmentation Model
AI总结：
本文针对MaskDINO中检测与分割性能不平衡的问题，提出DI-MaskDINO模型，通过DI模块和BATO模块缓解不平衡，在COCO和BDD100K上显著提升了联合检测与分割性能。
摘要：
本文的动机源于一个有趣的现象：在研究MaskDINO（即联合检测与分割的SOTA模型）起始transformer解码器层的中间结果时，发现目标检测的性能落后于实例分割（即性能不平衡）。这一现象启发我们思考一个问题：transformer解码器起始层的性能不平衡是否会限制最终性能的上限？带着这个问题，我们进一步进行了定性和定量的预实验，验证了检测-分割不平衡问题对模型性能的负面影响。为了解决这一问题，本文提出了DI-MaskDINO模型，其核心思想是通过缓解检测-分割不平衡来提升最终性能。DI-MaskDINO通过将我们提出的De-Imbalance（DI）模块和Balance-Aware Tokens Optimization（BATO）模块配置到MaskDINO中实现。DI负责生成平衡感知查询，BATO利用平衡感知查询来引导初始特征token的优化。平衡感知查询和优化后的特征token分别作为transformer解码器的Query和Key&Value，用于执行联合目标检测和实例分割。DI-MaskDINO在COCO和BDD100K基准上优于现有的联合目标检测和实例分割模型，与SOTA联合检测和分割模型MaskDINO相比，实现了+1.2 AP^{box}和+0.9 AP^{mask}的提升。此外，与SOTA目标检测模型DINO相比，DI-MaskDINO还获得了+1.0 AP^{box}的提升；与SOTA分割模型Mask2Former相比，获得了+3.0 AP^{mask}的提升。
英文摘要：
This paper is motivated by an interesting phenomenon: the performance of object detection lags behind that of instance segmentation (i.e., performance imbalance) when investigating the intermediate results from the beginning transformer decoder layer of MaskDINO (i.e., the SOTA model for joint detection and segmentation). This phenomenon inspires us to think about a question: will the performance imbalance at the beginning layer of transformer decoder constrain the upper bound of the final performance? With this question in mind, we further conduct qualitative and quantitative pre-experiments, which validate the negative impact of detection-segmentation imbalance issue on the model performance. To address this issue, this paper proposes DI-MaskDINO model, the core idea of which is to improve the final performance by alleviating the detection-segmentation imbalance. DI-MaskDINO is implemented by configuring our proposed De-Imbalance (DI) module and Balance-Aware Tokens Optimization (BATO) module to MaskDINO. DI is responsible for generating balance-aware query, and BATO uses the balance-aware query to guide the optimization of the initial feature tokens. The balance-aware query and optimized feature tokens are respectively taken as the Query and Key&Value of transformer decoder to perform joint object detection and instance segmentation. DI-MaskDINO outperforms existing joint object detection and instance segmentation models on COCO and BDD100K benchmarks, achieving +1.2 $AP^{box}$ and +0.9 $AP^{mask}$ improvements compared to SOTA joint detection and segmentation model MaskDINO. In addition, DI-MaskDINO also obtains +1.0 $AP^{box}$ improvement compared to SOTA object detection model DINO and +3.0 $AP^{mask}$ improvement compared to SOTA segmentation model Mask2Former.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
