SLAP：基于部分最优传输的鱼类重识别选择性局部视觉-语言对齐
SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport
浏览论文内容
中文总结 AI 辅助
针对鱼类重识别中全局对齐易引入噪声的问题，提出基于POT的选择性局部视觉-语言对齐框架，在多数据集上实现优于CLIP类方法的性能，泛化性良好。
中文摘要 AI 辅助
鱼类个体重识别（ReID）是一种细粒度识别问题，其中身份判别线索通常局限于特定身体区域，而非均匀分布在整个动物身上。然而，近期基于CLIP的ReID方法主要依赖全局图像-文本对齐，导致背景和弱判别区域也会对跨模态监督产生贡献。我们提出一种选择性局部视觉-语言对齐框架，该框架通过部分最优传输（Partial Optimal Transport, POT）建立视觉块嵌入与多个身份感知提示嵌入之间的局部对应关系。POT不强制建立详尽的对应，而是实现视觉块与提示嵌入之间的选择性匹配，使模型能够聚焦最强的跨模态对应，同时避免对弱匹配区域的强制对齐，从而生成更具判别性的视觉表示用于检索。该框架采用端到端训练，推理阶段仅保留适配后的视觉编码器。在纵向Symphodus melops数据集上的实验表明，该方法在闭集和开集评估协议下均优于近期基于CLIP的ReID方法；在其他数据集上的额外评估进一步证明了所提方法在各类海洋ReID基准上的泛化能力。
英文摘要
Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific body regions rather than distributed uniformly across the animal. Nevertheless, recent CLIP-based ReID methods rely predominantly on global image-text alignment, allowing background and weakly discriminative regions to contribute to cross-modal supervision. We propose a selective local vision-language alignment framework that establishes localized correspondences between visual patch embeddings and multiple identity-aware prompt embeddings through Partial Optimal Transport (POT). Rather than enforcing exhaustive correspondence, POT enables selective matching between visual patches and prompt embeddings, allowing the model to emphasize the strongest cross-modal correspondences while avoiding forced alignment of weakly matching regions, thereby yielding more discriminative visual representations for retrieval. The framework is trained end-to-end, while only the adapted visual encoder is retained during inference. Experiments on the longitudinal Symphodus melops dataset demonstrate consistent improvements over recent CLIP-based ReID methods under both closed-set and open-set evaluation protocols. Additional evaluations on other datasets further demonstrate the generalization capability of the proposed method across diverse marine ReID benchmarks.
发表机构
- University of Verona（维罗纳大学）
- Institute of Marine Research（海洋研究所）
- University of Agder（阿格德大学）
机构由 AI 辅助整理，请以论文原文为准。
