arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09850cs.CVcs.RO
收藏

MARVIS:运动与几何感知的真实与虚拟图像分割

MARVIS: Motion & Geometry Aware Real and Virtual Image Segmentation

University of Miami(迈阿密大学)

Jiayi Wu, Xiaomin Lin, Shahriar Negahdaripour, Cornelia Fermüller, Yiannis Aloimonos

AI总结:

针对海洋机器人中水面动态干扰导致感知失效问题,提出MARVIS方法,结合合成图像、运动熵核与极线几何一致性实现真实与虚拟图像分割,无需域重训练且性能优异。

摘要:

自主导航、3D重建和水面附近目标识别等任务在海洋机器人应用中至关重要。然而,动态干扰(如随机空气-水界面的光反射与折射、不规则液体流动及类似因素)会导致感知与导航系统潜在失效,带来挑战。传统计算机视觉算法难以区分真实与虚拟图像区域,显著增加任务复杂度。虚拟图像区域是光线重定向(通常通过反射或折射)形成的表观表示,营造物体存在的错觉却无实际物理位置。本工作提出一种新方法,利用合成图像结合域不变信息、运动熵核和极线几何一致性,实现真实与虚拟图像区域分割。该分割网络在域变化时无需重新训练,通过在仿真和现实两个不同域部署同一网络验证这一点。通过创建模拟水面复杂性的真实合成图像,为网络(MARVIS)提供细粒度训练数据,有效区分真实与虚拟图像。通过运动与几何感知的设计选择及全面实验分析,在未见过的现实域中实现最先进的真实-虚拟图像分割性能:IoU超过78%,F1-Score超过86%,同时保证较小计算量;MARVIS在单GPU(CPU核心)上的推理速率分别超过43 FPS(8 FPS)。代码和数据集可在https://github.com/jiayi-wu-umd/MARVIS获取。

英文摘要:

Tasks such as autonomous navigation, 3D reconstruction, and object recognition near the water surfaces are crucial in marine robotics applications. However, challenges arise due to dynamic disturbances, e.g., light reflections and refraction from the random air-water interface, irregular liquid flow, and similar factors, which can lead to potential failures in perception and navigation systems. Traditional computer vision algorithms struggle to differentiate between real and virtual image regions, significantly complicating tasks. A virtual image region is an apparent representation formed by the redirection of light rays, typically through reflection or refraction, creating the illusion of an object's presence without its actual physical location. This work proposes a novel approach for segmentation on real and virtual image regions, exploiting synthetic images combined with domain-invariant information, a Motion Entropy Kernel, and Epipolar Geometric Consistency. Our segmentation network does not need to be re-trained if the domain changes. We show this by deploying the same segmentation network in two different domains: simulation and the real world. By creating realistic synthetic images that mimic the complexities of the water surface, we provide fine-grained training data for our network (MARVIS) to discern between real and virtual images effectively. By motion & geometry-aware design choices and through comprehensive experimental analysis, we achieve state-of-the-art real-virtual image segmentation performance in unseen real world domain, achieving an IoU over 78% and a F1-Score over 86% while ensuring a small computational footprint. MARVIS offers over 43 FPS (8 FPS) inference rates on a single GPU (CPU core). Our code and dataset are available here https://github.com/jiayi-wu-umd/MARVIS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑