UVLM：面向水下世界理解的视频语言模型基准测试
UVLM: Benchmarking Video Language Model for Underwater World Understanding
AI总结：
针对现有视频语言模型缺乏水下场景适配基准的问题，本文构建了融合人类专家与AI协作的UVLM水下观测基准，通过多维度数据与任务设计验证了其可有效提升模型水下理解能力。
摘要：
近年来，大语言模型（LLMs）的显著成功对人工智能领域产生了深远影响，大量基于LLMs的先进工作被提出并应用于各类场景，其中视频语言模型（VidLMs）的应用尤为广泛。然而现有研究主要聚焦陆地场景，忽视了水下观测的高要求应用需求。为填补这一空白，我们推出了UVLM——一个通过人类专家与AI模型协作构建的水下观测基准。为保障数据质量，我们从多维度进行了深度考量：首先，针对水下环境的独特挑战，我们选取涵盖光照变化、水体浑浊、多样视角等典型水下难点的视频构建数据集；其次，为保证数据多样性，数据集覆盖了广泛的帧率、分辨率、419类海洋动物以及各类静态植物与地形；再者，在任务多样性方面，我们采用结构化设计，将观测目标划分为生物与环境两大类，每类均包含内容观测和变化/动作观测，共计20种不同任务类型；最后，我们设计了多项具有挑战性的评估指标，以实现不同方法的量化对比与分析。在两个代表性VidLMs上开展的实验表明，在UVLM上对VidLMs进行微调可显著提升其水下世界理解能力，同时在VideoMME、Perception text等现有空中场景VidLM基准上也展现出小幅性能提升的潜力。该数据集与提示工程将公开发布。
英文摘要：
Recently, the remarkable success of large language models (LLMs) has achieved a profound impact on the field of artificial intelligence. Numerous advanced works based on LLMs have been proposed and applied in various scenarios. Among them, video language models (VidLMs) are particularly widely used. However, existing works primarily focus on terrestrial scenarios, overlooking the highly demanding application needs of underwater observation. To overcome this gap, we introduce UVLM, an under water observation benchmark which is build through a collaborative approach combining human expertise and AI models. To ensure data quality, we have conducted in-depth considerations from multiple perspectives. First, to address the unique challenges of underwater environments, we selected videos that represent typical underwater challenges including light variations, water turbidity, and diverse viewing angles to construct the dataset. Second, to ensure data diversity, the dataset covers a wide range of frame rates, resolutions, 419 classes of marine animals, and various static plants and terrains. Next, for task diversity, we adopted a structured design where observation targets are categorized into two major classes: biological and environmental. Each category includes content observation and change/action observation, totaling 20 distinct task types. Finally, we designed several challenging evaluation metrics to enable quantitative comparison and analysis of different methods. Experiments on two representative VidLMs demonstrate that fine-tuning VidLMs on UVLM significantly improves underwater world understanding while also showing potential for slight improvements on existing in-air VidLM benchmarks, such as VideoMME and Perception text. The dataset and prompt engineering will be released publicly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
