The automation of workflows in advanced microscopy is a key goal where foundation models like Language Models (LLMs) and Vision-Language Models (VLMs) show great potential. However, adapting these general-purpose models for specialized scientific tasks is critical, and the optimal domain adaptation strategy is often unclear. To address this, we introduce PtychoBench, a new multi-modal, multi-task benchmark for ptychographic analysis. Using this benchmark, we systematically compare two specialization strategies: Supervised Fine-Tuning (SFT) and In-Context Learning (ICL). We evaluate these strategies on a visual artifact detection task with VLMs and a textual parameter recommendation task with LLMs in a data-scarce regime. Our findings reveal that the optimal specialization pathway is task-dependent. For the visual task, SFT and ICL are highly complementary, with a fine-tuned model guided by context-aware examples achieving the highest mean performance (Micro-F1 of 0.728). Conversely, for the textual task, ICL on a large base model is the superior strategy, reaching a peak Micro-F1 of 0.847 and outperforming a powerful "super-expert" SFT model (0-shot Micro-F1 of 0.839). We also confirm the superiority of context-aware prompting and identify a consistent contextual interference phenomenon in fine-tuned models. These results, benchmarked against strong baselines including GPT-4o and a DINOv3-based classifier, offer key observations for AI in science: the optimal specialization path in our benchmark is dependent on the task modality, offering a clear framework for developing more effective science-based agentic systems.
Adapting Foundation Model for Dental Caries Detection with Dual-View Co-Training
School of Biomedical Engineering \& State Key Laboratory of Advanced Medical Materials;Devices, ShanghaiTech University, Shanghai, China Lingang Laboratory, Shanghai, China Shanghai United Imaging Intelligence Co. Ltd., Shanghai, China Shanghai Clinical Research;Trial Center, Shanghai, China Shanghai Linkedcare Information Technology Co., Ltd., Shanghai, China
MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification
Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiaotong University(上海交通大学);Shanghai Ruijing Hospital, School of Medicine, Shanghai Jiao Tong University(上海交通大学医学院附属瑞金医院);Shanghai Tenth People’s Hospital of Tongji University(同济大学附属上海市第十人民医院);Xuzhou Central Hospital(徐州市中心医院);Renji Hospital, School of Medicine, Shanghai Jiao Tong University(上海交通大学医学院附属仁济医院);Sensetime Research(商汤科技研究院);The First Affiliated Hospital of Zhengzhou University(郑州大学第一附属医院);West China Hospital, Sichuan University(四川大学华西医院)
Cross-Domain Foundation Model Adaptation: Pioneering Computer Vision Models for Geophysical Data Analysis
School of Earth and Space Sciences, University of Science and Technology of China(中国科学技术大学地球和空间科学学院);Microsoft Applied Sciences Group(微软应用科学组);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室)
Adapting a Segmentation Foundation Model for Medical Image Classification
Department of Computer Science, University of Texas Rio Grande Valley(德克萨斯理工大学里奥格兰德分校计算机科学系);Department of Computer Science, Texas Woman’s University(德克萨斯女性大学计算机科学系);Sewickley Academy(塞维克利学院);Department of Mathematical Sciences, The University of Texas at Dallas(德克萨斯大学达拉斯分校数学科学系);Department of Computer Science and Engineering, University of Notre Dame(诺特丹大学计算机科学与工程系)
本文提出将 SAM 图像编码器作为冻结特征提取器,并设计空间局部化通道注意力机制 SLCA,将其集成到深度学习分类模型中,以增强对关键区域的关注,在三个公开医学图像分类数据集上验证了方法的有效性和数据高效性。
Adapting Vision Foundation Models with Cascaded Semantics
University of Alabama at Birmingham(阿拉巴马大学伯明翰分校);Carnegie Mellon University(卡内基梅隆大学);University of Missouri–Kansas City(密苏里大学堪萨斯城分校);Northeastern University(东北大学);Tulane University(杜兰大学);University of Bristol(布里斯托大学);Oak Ridge National Laboratory(橡树岭国家实验室)
VA-Adapter: Adapting Ultrasound Foundation Model to Echocardiography Probe Guidance
Department of Automation, BNRist, Tsinghua University, Beijing, China(自动化系、BNRist、清华大学、北京、中国);School of Computer Science and Technology, Xidian University(计算机科学与技术学院、西安电子科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Chinese PLA General Hospital(中国人民解放军总医院)
VFM-ISRefiner: Towards Better Adapting Vision Foundation Models for Interactive Segmentation of Remote Sensing Images
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航天信息研究所);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);School of Information Science and Technology, Beijing Forestry University(北京林业大学信息科学与技术学院)
Adapting Vision-Language Foundation Model for Next Generation Medical Ultrasound Image Analysis
Department of Health Technology and Informatics, The Hong Kong Polytechnic University(健康科技与信息学系,香港理工大学);Centre for Smart Health and School of Nursing, The Hong Kong Polytechnic University(智能健康中心及护理学院,香港理工大学);Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong(影像与介入放射学系,香港中文大学);Suzhou Hospital of Traditional Chinese Medicine Affiliated to Nanjing University of Chinese Medicine(南京中医药大学附属苏州中医医院);Department of Ultrasound, The Affiliated Changzhou No. 2 People's Hospital of Nanjing Medical University(南京医科大学附属常州第二人民医院超声科)
Adapting a Foundation Model for Lunar Surface Height Estimation
University of Technology of Troyes(特鲁瓦技术大学);Hochschule Darmstadt(达姆施塔特应用科学大学);European Space Agency(欧洲空间局);Technische Hochschule Würzburg-Schweinfurt(维尔茨堡-施韦因富特应用科学大学)
Block Expanded DINORET: Adapting Natural Domain Foundation Models for Retinal Imaging Without Catastrophic Forgetting
Stadtspital Zürich(苏黎世市立医院);Spross Research Institute(斯普罗斯研究所);Gutblick Research(古特布里克研究公司);Balgrist University Hospital(巴尔格里斯特大学医院);Moorfields Eye Hospital NHS Foundation Trust(摩尔菲尔德眼科医院国民保健服务基金会信托);Institute of Ophthalmology, University College London(伦敦大学学院眼科研究所);University College London(伦敦大学学院);National University of Singapore(新加坡国立大学);Singapore Eye Research Institute, Singapore National Eye Centre(新加坡国家眼科中心新加坡眼科研究所);Duke-NUS Medical School(杜克-新加坡国立大学医学院);iScreen 2 Prevent LLC(iScreen 2 Prevent有限责任公司);Bascom Palmer Eye Institute, University of Miami(迈阿密大学巴斯康帕默眼科研究所);University of Heidelberg(海德堡大学);Semmelweis University(塞梅尔魏斯大学)
Adapting Prithvi-EO for Fallow Detection for Food-Water Nexus: ViT-Adapter Necks and Parameter-Efficient Backbone tuning of Geospatial Foundation Model
Earth, Atmospheric and Geospatial Science, Saint Louis University(圣路易斯大学地球、大气与地理空间科学系)
T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoring
Institute of Artificial Intelligence(人工智能研究所);University of Stuttgart(斯图加特大学);Institute for Anthropomatics(人机学研究所);Karlsruhe Institute of Technology(卡尔斯鲁厄技术大学)
MM-DINOv2: Adapting Foundation Models for Multi-Modal Medical Image Analysis
Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM)(人工智能辅助影像诊断与治疗研究所,慕尼黑技术大学);TUM University Hospital, Munich, Germany(慕尼黑技术大学医院);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM)(人工智能在医疗与健康领域研究所,慕尼黑技术大学);Department of Radiation Oncology, TUM University Hospital, Munich, Germany(放射肿瘤科,慕尼黑技术大学医院);Chair for Computer Vision and Artificial Intelligence, Technical University of Munich (TUM)(计算机视觉与人工智能研究所,慕尼黑技术大学);Department of Scientific Computing, Florida State University(科学计算系,佛罗里达州立大学);Deutsches Konsortium für Translationale Krebsforschung (DKTK), Partner Site Munich(德国转化癌症研究联盟(DKTK)慕尼黑分部);Institute of Radiation Medicine (IRM), Department of Radiation Sciences (DRS), Helmholtz Center Munich(放射医学研究所(IRM),辐射科学部门(DRS),海德堡中心慕尼黑);Institute for Advanced Study, Technical University of Munich (TUM)(高级研究所,慕尼黑技术大学)
Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning
Mallinckrodt Institute of Radiology(马林克罗德放射医学研究所);Institute for Informatics, Data Science and Biostatistics(信息学、数据科学与生物统计学研究所);Department of Radiology, Perelman School of Medicine(放射科,佩尔曼医学院);Center for AI and Data Science for Integrated Diagnostics (AI2D)(集成诊断人工智能与数据科学中心(AI2D))
Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Experts
PCA Lab, Nanjing University of Science and Technology, Nanjing, China(南京理工大学科学与工程学院实验室);PCA Lab, VCIP, College of Computer Science, Nankai University, Tianjin, China(南开大学计算机学院)
Med-LEGO: Editing and Adapting toward Generalist Medical Image Diagnosis
School of Biomedical Engineering \& State Key Laboratory of Advanced Medical Materials;Devices, ShanghaiTech University, Shanghai, 201210, China Shanghai Clinical Research;Trial Center, Shanghai, 201210, China
EndoARSS: Adapting Spatially-Aware Foundation Model for Efficient Activity Recognition and Semantic Segmentation in Endoscopic Surgery
Department of Electronic Engineering, The Chinese University of Hong Kong(香港中文大学电子工程系);Shenzhen Research Institute, The Chinese University of Hong Kong(香港中文大学深圳研究院);School of Advanced Manufacturing, Fuzhou University(福州大学先进制造学院)