SmokeBench: 评估多模态大语言模型用于野火烟雾检测
SmokeBench: Evaluating Multimodal Large Language Models for Wildfire Smoke Detection
AI总结：
SmokeBench评估多模态大语言模型在野火烟雾检测中的性能，发现模型在烟雾定位方面存在显著局限，尤其在早期阶段。
摘要：
野火烟雾是透明、无定形的，并且常常在视觉上与云混淆，使得早期阶段的检测特别具有挑战性。在本工作中，我们介绍了一个基准，称为SmokeBench，用于评估多模态大语言模型（MLLMs）在图像中识别和定位野火烟雾的能力。该基准包含四个任务：（1）烟雾分类，（2）基于瓷砖的烟雾定位，（3）基于网格的烟雾定位，（4）烟雾检测。我们评估了几种MLLMs，包括Idefics2、Qwen2.5-VL、InternVL3、Unified-IO 2、Grounding DINO、GPT-4o和Gemini-2.5 Pro。我们的结果表明，尽管某些模型在烟雾覆盖大面积时可以分类烟雾的存在，但所有模型在准确定位方面都存在困难，尤其是在早期阶段。进一步分析发现，烟雾体积与模型性能有很强的相关性，而对比度在相对较不重要的角色。这些发现突显了当前MLLMs在安全关键性野火监测中的关键限制，并强调了需要改进早期烟雾定位方法的必要性。
英文摘要：
Wildfire smoke is transparent, amorphous, and often visually confounded with clouds, making early-stage detection particularly challenging. In this work, we introduce a benchmark, called SmokeBench, to evaluate the ability of multimodal large language models (MLLMs) to recognize and localize wildfire smoke in images. The benchmark consists of four tasks: (1) smoke classification, (2) tile-based smoke localization, (3) grid-based smoke localization, and (4) smoke detection. We evaluate several MLLMs, including Idefics2, Qwen2.5-VL, InternVL3, Unified-IO 2, Grounding DINO, GPT-4o, and Gemini-2.5 Pro. Our results show that while some models can classify the presence of smoke when it covers a large area, all models struggle with accurate localization, especially in the early stages. Further analysis reveals that smoke volume is strongly correlated with model performance, whereas contrast plays a comparatively minor role. These findings highlight critical limitations of current MLLMs for safety-critical wildfire monitoring and underscore the need for methods that improve early-stage smoke localization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
