MARINE：一种用于检测动物视频中罕见捕食者—猎物互动的计算机视觉模型
MARINE: A Computer Vision Model for Detecting Rare Predator-Prey Interactions in Animal Videos
AI总结：
本文提出 MARINE 视觉模型，结合运动帧选择与 DINOv2 特征，提升动物视频中罕见捕食行为的识别与时序检测性能。
摘要：
捕食者与猎物之间的遭遇在生态系统中发挥着重要作用，但其罕见性使其难以在视频记录中被检测到。尽管动作识别（AR）和时序动作检测（AD）取得了进展，尤其是基于 transformer 的模型和视觉基础模型已在人类动作数据集上实现高性能，但动物视频仍相对缺乏研究。本论文通过提出 MARINE 模型来弥补这一空白，该模型采用为快速动物动作设计的基于运动的帧选择，以及带有可训练分类头的 DINOv2 特征提取来进行动作识别。在识别鱼类视频中的捕食者攻击方面，MARINE 的表现优于 VideoMAE：在一个小型且特定的珊瑚礁数据集上，准确率为81.53%，而 VideoMAE 为52.64%；在规模更大的 Animal Kingdom 数据集的一个子集上，准确率为94.86%，而 VideoMAE 为83.14%。在 Animal Kingdom 代表性样本的多标签设置中，MARINE 达到23.79% mAP，在现有基准中处于中游。此外，在珊瑚礁数据集的一项 AD 任务中，尽管 t-IoU 阈值降低至25%，MARINE 仍取得80.78% AP，而 VideoMAE 为34.89%。因此，尽管仍有改进空间，MARINE 提供了一个有效的入门框架，可应用于动物记录的 AR 和 AD 任务，从而为自然生态系统研究作出贡献。
英文摘要：
Encounters between predator and prey play an essential role in ecosystems, but their rarity makes them difficult to detect in video recordings. Although advances in action recognition (AR) and temporal action detection (AD), especially transformer-based models and vision foundation models, have achieved high performance on human action datasets, animal videos remain relatively under-researched. This thesis addresses this gap by proposing the model MARINE, which utilizes motion-based frame selection designed for fast animal actions and DINOv2 feature extraction with a trainable classification head for action recognition. MARINE outperforms VideoMAE in identifying predator attacks in videos of fish, both on a small and specific coral reef dataset (81.53\% against 52.64\% accuracy), and on a subset of the more extensive Animal Kingdom dataset (94.86\% against 83.14\% accuracy). In a multi-label setting on a representative sample of Animal Kingdom, MARINE achieves 23.79\% mAP, positioning it mid-field among existing benchmarks. Furthermore, in an AD task on the coral reef dataset, MARINE achieves 80.78\% AP (against VideoMAE's 34.89\%) although at a lowered t-IoU threshold of 25\%. Therefore, despite room for improvement, MARINE offers an effective starter framework to apply to AR and AD tasks on animal recordings and thus contribute to the study of natural ecosystems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
