UW-VOS:一个大规模的水下视频物体分割数据集
Video Object Segmentation under Aquatic Domain Shift: A Diagnostic Benchmark and Restoration Adapter
AI总结:
本文提出UW-VOS数据集,用于水下视频物体分割,包含1431个视频序列和309295个掩码标注,并提出SAM-U框架在水下领域实现最优性能。
摘要:
水下视频物体分割(VOS)对于海洋探索至关重要,但开放环境的方法因颜色失真、对比度低和普遍的伪装而遭受显著退化。主要挑战是缺乏高质量的训练数据。为弥补这一差距,我们引入$\textbf{UW-VOS}$,首个大规模的水下VOS基准,包含1,431个视频序列,涵盖409个类别,拥有309,295个掩码标注,通过半自动数据引擎构建,经过严格的人工验证。我们进一步提出$\textbf{SAM-U}$,一个参数高效的框架,将SAM2适应到水下领域。通过在图像编码器中插入轻量级适配器,SAM-U仅使用$\sim$2$\%$的可训练参数即可实现最先进的性能。广泛实验表明,现有方法在UW-VOS上平均出现13点$\mathcal{J}\&\mathcal{F}$下降,而SAM-U有效弥合了这一领域差距。详细的属性分析进一步识别出小目标、伪装和进出问题为关键瓶颈,为未来在鲁棒水下感知中的研究提供了路线图。
英文摘要:
Underwater video object segmentation (VOS) requires temporally consistent dense segmentation in scenes where water acts as a participating optical medium. Wavelength-dependent attenuation, scattering, turbidity, and unstable illumination alter object appearance, weaken mask-level localization cues, and destabilize temporal association, yet existing VOS benchmarks provide limited support for diagnosing these aquatic factors. We introduce UW-VOS, a large-scale diagnostic benchmark with 1,431 video sequences, 307,267 mask annotations, 411 semantic categories, and 16 fine-grained attributes. UW-VOS supports both standard VOS evaluation and factorized analysis of aquatic appearance degradation, spatial localization ambiguity, and temporal identity discontinuity. We further propose the Aquatic Restoration Adapter (ARA), a physics-derived parameter-efficient module that transfers the inverse structure of underwater image formation into bottleneck feature space and can be instantiated in different VOS backbones. Experiments show that representative open-air VOS methods suffer an average 13-point $\mathcal{J}\&\mathcal{F}$ drop on UW-VOS. ARA matches or surpasses full fine-tuning in both Transformer- and memory-based VOS backbones, and outperforms standard PEFT counterparts under a matched setting. These results establish UW-VOS as a diagnostic platform for underwater video perception and ARA as a compact feature-restoration module for aquatic domain adaptation. Project repository: https://github.com/hszh/UW-VOS.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。