基于视觉语言模型的X射线荧光（XRF）与光学显微镜视场（FOV）定位

中文摘要 AI 辅助

对不同显微模态获取的图像进行配准，是关联同一样本互补测量的关键。在关联X射线荧光（XRF）与光学显微技术中，XRF图谱通常仅覆盖同一或相邻组织切片的光学图像的小区域，视场（FOV）定位是必要但因模态间外观与结构差异而颇具挑战的任务。本文在两组数据集上评估无需训练的视觉语言模型（VLM）定位性能，两组数据集分别代表同切片高对应度成像与相邻切片低对应度成像。我们测试了无约束搜索与元数据约束搜索，将VLM与几何对照方法、经典模板匹配及两种替代的无需训练方法（DINOv2与multiGradICON）进行对比。直接VLM提示产生了依赖内容的空间信号，但单独使用不可靠；经典匹配在跨模态结构保留时精度最高，但在低对应度数据集上失效；本文提出的“候选生成-验证”工作流，利用重复VLM预测作为候选，结合基于图像的相似性选择最终位置，该工作流在低对应度场景中实现了有效的定位。

英文摘要

Registering images acquired with different microscopy modalities is essential for relating complementary measurements of the same specimen. In correlative X-ray fluorescence (XRF) and optical microscopy, the XRF map often covers only a small region of an optical image acquired from the same or an adjacent tissue section. Field-of-view (FOV) localization is necessary but can be difficult when appearance and structure differ across modalities. Here we evaluate training-free vision language model (VLM) localization on two datasets representing same-section high-correspondence and adjacent-section low-correspondence imaging. We test unconstrained and metadata-constrained search and compare VLMs with geometric controls, classical template matching, and two alternative training-free approaches (DINOv2 and multiGradICON). Direct VLM prompting produced content-dependent spatial signals but was not reliable alone. Classical matching was most accurate when cross-modal structure was preserved but failed in the low-correspondence collection. A proposal-and-verify workflow used repeated VLM predictions as candidates and image-based similarity to select the final location. This workflow recovered useful localization in the low-correspondence regime.

发表机构

• Northwestern University（西北大学）
• University of Chicago（芝加哥大学）
• Oregon Health and Science University（俄勒冈健康与科学大学）
• Argonne National Laboratory（阿贡国家实验室）

机构由 AI 辅助整理，请以论文原文为准。