用于跨模态水下机器人感知的声纳-视觉数据集
A Sonar-Visual Dataset for Cross-Modal Underwater Robot Perception
AI总结：
提出SOVIS数据集，包含76,000多对声纳-视觉帧，通过端到端管道同步和清洗数据，并利用交互式标注工具加速标注，在跨模态鱼类检测任务中实现mAP@0.10提升7倍。
摘要：
水下机器人通常同时使用相机和声纳进行感知，以利用视觉丰富的语义细节和声学稳健的距离测量。然而，由于缺乏声纳-视觉配对数据集，通过跨模态预测学习这些模态之间的映射仍然探索不足。我们提出了SOVIS，一个用于跨模态水下感知的声纳-视觉数据集。SOVIS包含在特隆赫姆峡湾六个地点17次潜水中收集的超过76,000对配对帧，并得到端到端管道的支持，该管道清洁和同步跨模态传感器数据。我们还引入了一个交互式标注工具，旨在加速配对数据的标注过程。最后，我们使用一小部分标注数据展示了一个概念验证的跨模态鱼类检测任务，与单目相机基线相比，mAP@0.10提高了7倍。SOVIS是推进跨模态水下感知研究的第一步，支持从单目图像进行密集声纳预测等研究方向。
英文摘要：
Underwater robots typically use both cameras and sonar for perception to leverage the rich semantic details of vision and the robust range measurements of acoustics. However, learning to map between these modalities via cross-modal prediction remains underexplored due to limited sonar-visual paired datasets. We present SOVIS, a sonar-visual dataset for cross-modal underwater perception. SOVIS comprises over 76,000 paired frames collected across 17 dives at six sites in the Trondheimfjord, supported by an end-to-end pipeline that cleans and synchronizes the cross-modal sensor data. We also introduce an interactive annotation tool designed to accelerate the labeling process for this paired data. Finally, we demonstrate a proof-of-concept cross-modal fish detection task using a small subset of labeled data, achieving a 7x improvement in mAP@0.10 over a monocular camera baseline. SOVIS serves as the first step toward advancing cross-modal underwater perception research, enabling research directions such as dense sonar prediction from monocular images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
