光学显微镜中用于少样本细胞检测的视觉语言模型上下文自适应
In-Context Adaptation of VLMs for Few-Shot Cell Detection in Optical Microscopy
AI总结：
本文提出Micro-OD基准并研究视觉语言模型的上下文少样本细胞检测能力，通过混合检测流程结合VLM分类器提升显微镜图像中的少样本目标检测性能。
摘要：
基础视觉语言模型（VLMs）在自然图像上表现优异，但它们在生物医学显微镜中的实用性仍未得到充分探索。在本文中，我们研究了在大规模标注数据集不可用的情况下（显微镜图像常常如此），上下文学习如何使最先进的VLMs能够执行少样本目标检测。我们引入了Micro-OD基准，这是一个专门为上下文学习而精心整理的包含252张图像的数据集，其边界框标注涵盖来自四个来源的11种细胞类型，其中包括两个实验室内部专家标注的数据集。我们系统性地在少样本条件下评估了八个VLMs，并比较了带有和不带有隐式测试时推理token的变体。我们进一步实现了一个混合少样本目标检测（FSOD）流程，该流程将检测头与基于VLM的少样本分类器相结合，从而在我们基准上提升了近期VLMs的少样本性能。在各个数据集上，我们观察到由于领域差距，零样本性能较弱；然而，少样本支持持续改善了检测效果，在六次样本之后获得的边际增益。我们观察到带有推理token的模型在端到端定位方面更有效，而更简单的变体更适合对预先定位的图像块进行分类。我们的结果凸显了上下文自适应作为显微镜应用的一条实用路径，并且我们的基准为推进生物医学成像中的开放词汇检测提供了一个可复现的测试平台。
英文摘要：
Foundation vision-language models (VLMs) excel on natural images, but their utility for biomedical microscopy remains underexplored. In this paper, we investigate how in-context learning enables state-of-the-art VLMs to perform few-shot object detection when large annotated datasets are unavailable, as is often the case with microscopic images. We introduce the Micro-OD benchmark, a curated collection of 252 images specifically curated for in-context learning, with bounding-box annotations spanning 11 cell types across four sources, including two in-lab expert-annotated sets. We systematically evaluate eight VLMs under few-shot conditions and compare variants with and without implicit test-time reasoning tokens. We further implement a hybrid Few-Shot Object Detection (FSOD) pipeline that combines a detection head with a VLM-based few-shot classifier, which enhances the few-shot performance of recent VLMs on our benchmark. Across datasets, we observe that zero-shot performance is weak due to the domain gap; however, few-shot support consistently improves detection, with marginal gains achieved after six shots. We observe that models with reasoning tokens are more effective for end-to-end localization, whereas simpler variants are more suitable for classifying pre-localized crops. Our results highlight in-context adaptation as a practical path for microscopy, and our benchmark provides a reproducible testbed for advancing open-vocabulary detection in biomedical imaging.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
