WangLab at MEDIQA-M3G 2024: Multimodal Medical Answer Generation using Large Language Models
University of Toronto(多伦多大学);Vector Institute(向量研究所);Princess Margaret Cancer Centre, University Health Network(大学健康网络玛格丽特公主癌症中心);Lunenfeld-Tanenbaum Research Institute, Sinai Health System(西奈健康系统勒嫩菲尔德-塔南鲍姆研究所);Peter Munk Cardiac Centre, University Health Network(大学健康网络彼得·芒克心脏中心);AI Hub, University Health Network(大学健康网络人工智能中心)
Ronald Xie, Steven Palayew, Augustin Toma, Gary Bader, Bo Wang
本文介绍了WangLab参加MEDIQA-M3G 2024共享任务的两个方案,分别基于Claude 3 Opus API调用和CLIP风格图像分类,在英语类别中取得第一和第二名,并指出这两种方法在医学视觉问答中具有进一步研究潜力。
摘要:
本文概述了我们提交给MEDIQA2024多语言和多模态医学答案生成(M3G)共享任务的方案。我们报告了该任务英语类别下两个独立解决方案的结果:第一个方案涉及对Claude 3 Opus API进行两次连续的API调用;第二个方案涉及以CLIP的风格训练图像-疾病标签联合嵌入,用于图像分类。这两个解决方案在竞赛排行榜上分别获得第1名和第2名,大幅优于次优方案。此外,我们讨论了赛后实验中获得的见解。尽管由于共享任务的难度以及医学视觉问答普遍具有的挑战性,这两个解决方案的性能仍有很大的提升空间,但我们认为多阶段LLM方法和CLIP图像分类方法是值得进一步研究的途径。
英文摘要:
This paper outlines our submission to the MEDIQA2024 Multilingual and Multimodal Medical Answer Generation (M3G) shared task. We report results for two standalone solutions under the English category of the task, the first involving two consecutive API calls to the Claude 3 Opus API and the second involving training an image-disease label joint embedding in the style of CLIP for image classification. These two solutions scored 1st and 2nd place respectively on the competition leaderboard, substantially outperforming the next best solution. Additionally, we discuss insights gained from post-competition experiments. While the performance of these two solutions have significant room for improvement due to the difficulty of the shared task and the challenging nature of medical visual question answering in general, we identify the multi-stage LLM approach and the CLIP image classification approach as promising avenues for further investigation.
WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction
University of Toronto(多伦多大学);Vector Institute(矢量研究所);McGill University(麦吉尔大学);Peter Munk Cardiac Centre, University Health Network(大学健康网络彼得·芒克心脏中心);AI Hub, University Health Network(大学健康网络人工智能中心)
WangLab at MEDIQA-Chat 2023: Clinical Note Generation from Doctor-Patient Conversations using Large Language Models
University of Toronto(多伦多大学);Terrence Donnelly Centre for Cellular and Biomolecular Research(特伦斯·唐纳利细胞与生物分子研究中心);Vector Institute for AI(向量人工智能研究院);University Health Network(大学健康网络);Sunnybrook Health Sciences Centre(桑尼布鲁克健康科学中心)
Edinburgh Clinical NLP at MEDIQA-CORR 2024: Guiding Large Language Models with Hints
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Usher Institute, University of Edinburgh(爱丁堡大学厄舍研究所);Edinburgh Futures Institute, University of Edinburgh(爱丁堡未来研究所);School of Literatures, Languages and Cultures, University of Edinburgh(爱丁堡大学文学、语言与文化学院)
MediQAl: A French Medical Question Answering Dataset for Knowledge and Reasoning Evaluation
Data Clinic, University Hospital of Nantes, France(南特大学医院数据诊所,法国);Nantes Université, École Centrale Nantes, CNRS, LS2N, France(南特大学,中央理工学院南特分校,国家科学研究中心,LS2N,法国)
Dr.Quad at MEDIQA 2019: Towards Textual Inference and Question Entailment using contextualized representations
Carnegie Mellon University(卡内基梅隆大学)
本文介绍了Quad博士团队提交的ACL-BioNLP 2019共享任务,即医学领域的文本推理和问题蕴涵。我们的系统基于Liu et al.(2019)的先前工作,该工作使用多任务目标函数进行文本蕴涵。在这项工作中,我们探讨了不同的战略,推广国家的最先进的语言理解模型的专业医疗领域。我们在共享任务上的结果表明,通过数据增强整合领域知识是应对医学等专业领域所带来挑战的有力策略。