面向无遮蔽标注的零样本遮蔽物分割
Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations
AI总结：
本文提出了一种无需遮蔽标注的零样本遮蔽物分割框架，通过结合MIM、M-LLM和MFA机制，实现高效分割与快速推理。
摘要：
遮蔽物分割（COS）面临由于标注数据稀缺带来的重大挑战，其中细致的像素级标注既耗时又昂贵，主要由于物体与背景边界的复杂性。针对核心问题“能否在没有任何遮蔽物手动标注的情况下实现有效的零样本分割？”我们肯定回答并引入了一个稳健的零样本COS框架。该框架利用COS固有的局部模式偏差，并从显著物分割（SOS）中衍生出广泛语义特征空间，以实现高效的零样本迁移。我们结合基于Masked Image Modeling（MIM）的图像编码器，该编码器针对参数高效微调（PEFT）进行了优化，多模态大语言模型（M-LLM）以及多尺度细粒度对齐（MFA）机制。MIM预训练的图像编码器专注于捕捉本质的低级特征，而M-LLM生成的标题嵌入与这些视觉提示一起处理。这些嵌入通过MFA精确对齐，使我们的框架能够准确解释和导航复杂的语义上下文。为优化操作效率，我们引入了一个可学习的代码本，在推理过程中表示M-LLM，显著减少了计算开销。我们的框架通过严格的实验验证其通用性和有效性，在CAMO和COD10K上分别取得了72.9%和71.7%的F_β^w得分。通过在推理过程中移除M-LLM，我们实现了与传统端到端模型相当的推理速度，达到18.1 FPS。代码：https://github.com/AVC2-UESTC/ZSCOS-CaMF
英文摘要：
Camouflaged Object Segmentation (COS) faces significant challenges due to the scarcity of annotated data, where meticulous pixel-level annotation is both labor-intensive and costly, primarily due to the intricate object-background boundaries. Addressing the core question, "Can COS be effectively achieved in a zero-shot manner without manual annotations for any camouflaged object?" we affirmatively respond and introduce a robust zero-shot COS framework. This framework leverages the inherent local pattern bias of COS and employs a broad semantic feature space derived from salient object segmentation (SOS) for efficient zero-shot transfer. We incorporate an Masked Image Modeling (MIM) based image encoder optimized for Parameter-Efficient Fine-Tuning (PEFT), a Multimodal Large Language Model (M-LLM), and a Multi-scale Fine-grained Alignment (MFA) mechanism. The MIM pre-trained image encoder focuses on capturing essential low-level features, while the M-LLM generates caption embeddings processed alongside these visual cues. These embeddings are precisely aligned using MFA, enabling our framework to accurately interpret and navigate complex semantic contexts. To optimize operational efficiency, we introduce a learnable codebook that represents the M-LLM during inference, significantly reducing computational overhead. Our framework demonstrates its versatility and efficacy through rigorous experimentation, achieving state-of-the-art performance in zero-shot COS with $F_β^w$ scores of 72.9\% on CAMO and 71.7\% on COD10K. By removing the M-LLM during inference, we achieve an inference speed comparable to that of traditional end-to-end models, reaching 18.1 FPS. Code: https://github.com/AVC2-UESTC/ZSCOS-CaMF
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
