arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.10568cs.CV
收藏

AgMMU:一个综合性农业多模态理解基准

AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Rice University(Rice大学);Carnegie Mellon University(卡内基梅隆大学);AIFARMS;Center for Digital Agriculture at UIUC(伊利诺伊大学厄巴纳-香槟分校数字农业中心)

Aruna Gauba, Irene Pi, Yunze Man, Ziqi Pang, Vikram S. Adve, Yu-Xiong Wang

AI总结:

该研究基于真实农业专家对话构建 AgMMU 多模态评测基准与 AgBase 知识语料,揭示主流 VLM 的农业知识差距,并证明微调可显著提升开源模型表现。

摘要:

我们提出 AgMMU,这是一个具有挑战性的真实世界基准,用于在知识密集型农业领域评估和推进视觉-语言模型(VLM)。与以往依赖众包提示的数据集不同,AgMMU 从日常种植者与 USDA 授权的合作推广专家之间的 116,231 段真实对话中提炼而来。通过自动化知识抽取、问答生成和人工验证这一三阶段流程,我们构建了:(i)AgMMU,一个包含 746 道多项选择题(MCQ)和 746 道开放式问题(OEQ)的评估集;以及(ii)AgBase,一个涵盖 57,079 条多模态事实的开发语料库,覆盖五个高风险农业主题:昆虫识别、物种识别、病害分类、症状描述和管理指导。对 12 个领先 VLM 的基准测试揭示了它们在细粒度感知和事实依据方面的明显差距。开源模型大幅落后于专有模型。在 AgBase 上进行简单微调,可使开源模型在具有挑战性的 OEQ 上的性能平均最高提升 11.6%,从而缩小这一差距,并激励未来研究提出更好的策略,以从 AgBase 中进行知识抽取和蒸馏。我们希望 AgMMU 能够推动农业 AI 发展中关于领域特定知识整合和可信决策支持的研究。

英文摘要:

We present AgMMU, a challenging real-world benchmark for evaluating and advancing vision-language models (VLMs) in the knowledge-intensive domain of agriculture. Unlike prior datasets that rely on crowdsourced prompts, AgMMU is distilled from 116,231 authentic dialogues between everyday growers and USDA-authorized Cooperative Extension experts. Through a three-stage pipeline: automated knowledge extraction, QA generation, and human verification, we construct (i) AgMMU, an evaluation set of 746 multiple-choice questions (MCQs) and 746 open-ended questions (OEQs), and (ii) AgBase, a development corpus of 57,079 multimodal facts covering five high-stakes agricultural topics: insect identification, species identification, disease categorization, symptom description, and management instruction. Benchmarking 12 leading VLMs reveals pronounced gaps in fine-grained perception and factual grounding. Open-sourced models trail after proprietary ones by a wide margin. Simple fine-tuning on AgBase boosts open-sourced model performance on challenging OEQs for up to 11.6% on average, narrowing this gap and also motivating future research to propose better strategies in knowledge extraction and distillation from AgBase. We hope AgMMU stimulates research on domain-specific knowledge integration and trustworthy decision support in agriculture AI development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑