arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.02503cs.CV
收藏

为低数据情形下的X射线衍射成像适应通用基础模型

Adapting General-Purpose Foundation Models for X-ray Ptychography in Low-Data Regimes

The University of Alabama(阿拉巴马大学);Argonne National Laboratory(阿贡国家实验室)

Robinson Umeike, Neil Getty, Yin Xiangyu, Yi Jiang

AI总结:

本文提出PtychoBench基准,通过比较SFT和ICL策略,在低数据环境下优化X射线衍射成像任务的模型适应性,发现任务模态决定最佳专门化路径。

摘要:

先进显微镜工作流的自动化是关键目标,基础模型如语言模型(LLMs)和视觉-语言模型(VLMs)显示出巨大潜力。然而,将这些通用模型适应于专门的科学任务是关键,而最佳的领域适应策略往往不明确。为此,我们引入PtychoBench,一个新的多模态、多任务基准,用于衍射分析。利用此基准,我们系统比较了两种专门化策略:监督微调(SFT)和上下文学习(ICL)。我们在数据稀缺情况下,用VLMs评估了在视觉伪影检测任务中的两种策略,用LLMs评估了在文本参数推荐任务中的两种策略。我们的发现表明,最佳专门化路径取决于任务。对于视觉任务,SFT和ICL高度互补,由上下文感知示例引导的微调模型在均值性能上最高(Micro-F1为0.728)。相反,对于文本任务,对大型基础模型进行ICL是更优策略,达到Micro-F1峰值0.847,优于强大的“超级专家”SFT模型(0次微调Micro-F1为0.839)。我们还确认了上下文感知提示的优越性,并识别了微调模型中一致的上下文干扰现象。这些结果,基于GPT-4o和基于DINOv3的分类器等强大基线,为科学中的AI提供了关键观察:在我们基准中的最佳专门化路径取决于任务模态,为开发更有效的基于科学的智能系统提供了清晰的框架。

英文摘要:

The automation of workflows in advanced microscopy is a key goal where foundation models like Language Models (LLMs) and Vision-Language Models (VLMs) show great potential. However, adapting these general-purpose models for specialized scientific tasks is critical, and the optimal domain adaptation strategy is often unclear. To address this, we introduce PtychoBench, a new multi-modal, multi-task benchmark for ptychographic analysis. Using this benchmark, we systematically compare two specialization strategies: Supervised Fine-Tuning (SFT) and In-Context Learning (ICL). We evaluate these strategies on a visual artifact detection task with VLMs and a textual parameter recommendation task with LLMs in a data-scarce regime. Our findings reveal that the optimal specialization pathway is task-dependent. For the visual task, SFT and ICL are highly complementary, with a fine-tuned model guided by context-aware examples achieving the highest mean performance (Micro-F1 of 0.728). Conversely, for the textual task, ICL on a large base model is the superior strategy, reaching a peak Micro-F1 of 0.847 and outperforming a powerful "super-expert" SFT model (0-shot Micro-F1 of 0.839). We also confirm the superiority of context-aware prompting and identify a consistent contextual interference phenomenon in fine-tuned models. These results, benchmarked against strong baselines including GPT-4o and a DINOv3-based classifier, offer key observations for AI in science: the optimal specialization path in our benchmark is dependent on the task modality, offering a clear framework for developing more effective science-based agentic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑