LithoBench：用于遥感岩石学解释的大型多模态模型基准测试
LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation
AI总结：
本文提出LithoBench，一个用于评估遥感岩石学解释中地质语义理解的多级基准，包含10000个专家标注实例，涵盖12种岩石类型，通过专家反馈的半自动化流程提升评估可靠性。
摘要：
遥感岩石学解释是地质调查、矿产勘探和区域地质制图的基础。与一般土地覆盖识别不同，岩石学解释是知识密集型任务，需要专家从各种特征中推断岩石类型，如细微的视觉、光谱、纹理、地貌和上下文线索，使可靠的自动化解释极具挑战性。地质知识引导的大型多模态模型提供了新机会，但其评估受限于缺乏能捕捉岩石学注释、多级地质语义和专家反馈的基准。本文提出LithoBench，一个用于评估遥感岩石学解释中地质语义理解的多级基准。LithoBench包含10000个专家标注的解释实例，涵盖12种代表性岩石类型，包括4000个多项选择题和6000个开放式任务，分为五个认知层次：识别与描述、比较分析、机制解释、实际应用和综合推理。我们进一步开发了专家在环、知识引导的半自动化构建流程，结合多子流程，如结构化的地质图像描述，以增强地质有效性和评估可靠性。对多种大型视觉-语言模型的实验显示，在地质语义理解方面存在显著局限性，尤其是在更高阶的解释、应用和推理任务上。
英文摘要：
Remote sensing lithology interpretation is fundamental to geological surveys, mineral exploration, and regional geological mapping. Unlike general land-cover recognition, lithology interpretation is a knowledge-intensive task that requires experts to infer rock types from various features, e.g., subtle visual, spectral, textural, geomorphological, and contextual cues, making reliable automated interpretation highly challenging. Geological knowledge-guided large multimodal models offer new opportunities, yet their evaluation remains constrained by the lack of benchmarks that capture lithological annotations, multi-level geological semantics, and expert-informed assessment. Here, we propose LithoBench, a multi-level benchmark for evaluating geological semantic understanding in remote sensing lithology interpretation. LithoBench contains 10,000 expert-annotated interpretation instances across 12 representative lithological categories, including 4,000 multiple-choice and 6,000 open-ended tasks organized into five cognitive levels: Identification and Description, Comparative Analysis, Mechanism Explanation, Practical Application, and Comprehensive Reasoning. We further develop an expert-in-the-loop, knowledge-grounded semi-automated construction pipeline, coupling multi sub-processes, e.g., structured geological image descriptions, to enhance geological validity and evaluation reliability. Experiments with multiple large vision-language models eveal substantial limitations in geological semantic understanding, particularly on higher-order explanation, application, and reasoning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
