μ-Bench：面向显微图像理解的视觉-语言基准
μ-Bench: A Vision-Language Benchmark for Microscopy Understanding
AI总结：
针对显微图像理解缺乏标准化评测的问题，本文构建涵盖22项跨学科任务的μ-Bench，评估多类VLM并揭示微调导致灾难性遗忘，提出权重插值缓解方案。
摘要：
显微技术的最新进展使得细胞生物学和生物医学研究中能够快速生成太字节级的图像数据。视觉-语言模型（VLM）为大规模生物图像分析提供了一种有前景的解决方案，可提高研究人员的效率、识别新的图像生物标志物，并加速假设生成与科学发现。然而，目前缺乏标准化、多样化且大规模的视觉-语言基准，用于评估VLM在生物图像理解中的感知与认知能力。为填补这一空白，我们提出μ-Bench，这是一个由专家整理的基准，涵盖跨多个学科（生物学、病理学）、显微模态（电子、荧光、光学）、尺度（亚细胞、细胞、组织）以及正常和异常状态下生物体的22项生物医学任务。我们在μ-Bench上评估了最先进的生物医学、病理学和通用VLM，发现：i）当前模型在所有类别上都表现吃力，即使是区分显微模态等基础任务；ii）当前在生物医学数据上微调的专用模型往往比通用模型表现更差；iii）在特定显微领域进行微调可能导致灾难性遗忘，削弱其基础模型中已编码的既有生物医学知识；iv）在微调模型与预训练模型之间进行权重插值，为遗忘问题提供了一种解决方案，并提升了跨生物医学任务的总体性能。我们以宽松许可协议发布μ-Bench，以加速显微基础模型的研究与开发。
英文摘要：
Recent advances in microscopy have enabled the rapid generation of terabytes of image data in cell biology and biomedical research. Vision-language models (VLMs) offer a promising solution for large-scale biological image analysis, enhancing researchers' efficiency, identifying new image biomarkers, and accelerating hypothesis generation and scientific discovery. However, there is a lack of standardized, diverse, and large-scale vision-language benchmarks to evaluate VLMs' perception and cognition capabilities in biological image understanding. To address this gap, we introduce μ-Bench, an expert-curated benchmark encompassing 22 biomedical tasks across various scientific disciplines (biology, pathology), microscopy modalities (electron, fluorescence, light), scales (subcellular, cellular, tissue), and organisms in both normal and abnormal states. We evaluate state-of-the-art biomedical, pathology, and general VLMs on μ-Bench and find that: i) current models struggle on all categories, even for basic tasks such as distinguishing microscopy modalities; ii) current specialist models fine-tuned on biomedical data often perform worse than generalist models; iii) fine-tuning in specific microscopy domains can cause catastrophic forgetting, eroding prior biomedical knowledge encoded in their base model. iv) weight interpolation between fine-tuned and pre-trained models offers one solution to forgetting and improves general performance across biomedical tasks. We release μ-Bench under a permissive license to accelerate the research and development of microscopy foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
