Department of Biomedical Engineering, Faculty of Engineering, Mahidol University(生物医学工程系,工程学院,玛希多大学);Faculty of Medicine Ramathibodi Hospital, Mahidol University(拉玛蒂波迪医院医学院,玛希多大学)
Scientific figure captions require both accuracy and stylistic consistency to convey visual information. Here, we present a domain-specific caption generation system for the 3rd SciCap Challenge that integrates figure-related textual context with author-specific writing styles using the LaMP-Cap dataset. Our approach uses a two-stage pipeline: Stage 1 combines context filtering, category-specific prompt optimization via DSPy's MIPROv2 and SIMBA, and caption candidate selection; Stage 2 applies few-shot prompting with profile figures for stylistic refinement. Our experiments demonstrate that category-specific prompts outperform both zero-shot and general optimized approaches, improving ROUGE-1 recall by +8.3\% while limiting precision loss to -2.8\% and BLEU-4 reduction to -10.9\%. Profile-informed stylistic refinement yields 40--48\% gains in BLEU scores and 25--27\% in ROUGE. Overall, our system demonstrates that combining contextual understanding with author-specific stylistic adaptation can generate captions that are both scientifically accurate and stylistically faithful to the source paper.
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023
Pennsylvania State University(宾夕法尼亚州立大学);National Tsing Hua University(国立清华大学);AllSci(奥尔赛公司);MetaMetrics Inc.(美塔麦特里克公司);Adobe Research(奥多比研究院);Institute of Information Science, Academia Sinica(中央研究院资讯科学研究所)
MedICaT: A Dataset of Medical Images, Captions, and Textual References
University of Washington(华盛顿大学);Tel Aviv University(特拉维夫大学);Swedish Medical Group(瑞典医疗集团);University of California, Irvine(加州大学欧文分校);Allen Institute for AI(艾伦人工智能研究所)
Beyond Captions: Context-Grounded Reconstruction for Biomedical Multimodal Continued Pretraining
The Hong Kong Polytechnic University(香港理工大学);Sun Yat-sen University(中山大学);InfiX.ai;PolyU-Daya Bay Technology and Innovation Research Institute(PolyU-大亚湾技术与创新研究院)
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis
University of Mannheim(曼海姆大学);University of Technology Nuremberg(纽伦堡理工大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);National Institute of Information and Communications Technology(日本信息通信研究机构)
Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy
Energy, Materials and Environment Research Centre, London South Bank University, London SE1 0AA, UK(能源、材料与环境研究中心,伦敦南银行大学);School of Engineering and Design, London South Bank University, London SE1 0AA, UK(工程与设计学院,伦敦南银行大学);Bioscience and Bioengineering Research Centre, London South Bank University, London SE1 0AA, UK(生物科学与生物工程研究中心,伦敦南银行大学);Department of Physics, Kings College London, London WC2R 2LS, UK(物理系,伦敦国王学院)
How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures
University of Aberdeen(阿伯丁大学);International Institute of Information Technology Hyderabad(海德拉巴国际信息技术学院);University of Technology Nuremberg(纽伦堡工业大学);University of Southern California(南加利福尼亚大学)