arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.04593cs.CVcs.IR
收藏

MarineVRS:通过语义理解实现可解释性的海洋视频检索系统

MarineVRS: Marine Video Retrieval System with Explainability via Semantic Understanding

The Hong Kong University of Science and Technology(香港科技大学)

Tan-Sang Ha, Hai Nguyen-Truong, Tuan-Anh Vu, Sai-Kit Yeung

AI总结:

本文提出面向海洋领域的 MarineVRS,融合视觉与语言表征实现水下视频检索,并通过可解释性模块输出查询对象分割掩码,提升分析准确性与可扩展性。

摘要:

构建一个稳健可靠的视频检索系统(尤其是面向海洋环境)是一项具有挑战性的任务,原因涉及多个因素,例如需要处理海量密集且重复的数据、遮挡、模糊、低光照条件以及抽象查询。为应对这些挑战,我们提出了 MarineVRS,这是一种新颖且灵活的、专门为海洋领域设计的视频检索系统。MarineVRS 集成了用于视觉和语言对象表征的最先进方法,以实现对大量水下视频数据的高效、准确搜索与分析。此外,传统视频检索系统仅允许用户对图像或视频集合建立索引,并使用自由形式的自然语言句子进行搜索;与这类系统不同,我们的检索系统额外包含一个可解释性模块,该模块输出输入查询所指对象的分割掩码。此功能使用户能够识别并分离视频片段中的特定对象,从而对其行为和运动进行更细致的分析与理解。最后,凭借其适应性、可解释性、准确性和可扩展性,MarineVRS 成为海洋研究人员和科学家高效、准确地处理海量数据,并更深入洞察海洋物种行为与运动的强大工具。

英文摘要:

Building a video retrieval system that is robust and reliable, especially for the marine environment, is a challenging task due to several factors such as dealing with massive amounts of dense and repetitive data, occlusion, blurriness, low lighting conditions, and abstract queries. To address these challenges, we present MarineVRS, a novel and flexible video retrieval system designed explicitly for the marine domain. MarineVRS integrates state-of-the-art methods for visual and linguistic object representation to enable efficient and accurate search and analysis of vast volumes of underwater video data. In addition, unlike the conventional video retrieval system, which only permits users to index a collection of images or videos and search using a free-form natural language sentence, our retrieval system includes an additional Explainability module that outputs the segmentation masks of the objects that the input query referred to. This feature allows users to identify and isolate specific objects in the video footage, leading to more detailed analysis and understanding of their behavior and movements. Finally, with its adaptability, explainability, accuracy, and scalability, MarineVRS is a powerful tool for marine researchers and scientists to efficiently and accurately process vast amounts of data and gain deeper insights into the behavior and movements of marine species.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑