Building an intelligent and efficient medical assistant is still a challenging AI problem. The major limitation comes from the data modality scarceness, which reduces comprehensive patient perception. This demo paper presents the GigaPevt, the first multimodal medical assistant that combines the dialog capabilities of large language models with specialized medical models. Such an approach shows immediate advantages in dialog quality and metric performance, with a 1.18% accuracy improvement in the question-answering task.
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
ZJU-UIUC Institute, Zhejiang University(浙江大学ZJU-UIUC学院);University of Washington(华盛顿大学);Zhejiang University School of Medicine Sir Run Run Shaw Hospital(浙江大学医学院附属邵逸夫医院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applications
Academy for Engineering and Technology, Fudan University(复旦大学工程与技术学院);Tencent Youtu Lab(腾讯优图实验室);Cognition and Intelligent Technology Laboratory(认知与智能技术实验室);Engineering Research Center of AI and Robotics, Ministry of Education(教育部人工智能与机器人工程研究中心);AI and Unmanned Systems Engineering Research Center of Jilin Province(吉林省人工智能与无人系统工程研究中心)
RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models
School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);School of Computing, Clemson University(克莱姆森大学计算机学院);Washington University School of Medicine in St. Louis(圣路易斯华盛顿大学医学院);The Keck School of Medicine of the University of Southern California(南加州大学凯克医学院);Mayo Clinic(梅奥诊所);Banner Alzheimer’s Institute(班纳阿尔茨海默病研究所)
Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian
National University of Science and Technology POLITEHNICA Bucharest(波兰技术科学大学);Dalle Molle Institute for Artificial Intelligence Research (IDSIA)(达勒莫利人工智能研究所);MedicChat
EyeGPT: Ophthalmic Assistant with Large Language Models
The Hong Kong Polytechnic University(香港理工大学);Centre for Eye and Vision Research (CEVR)(眼科与视觉研究中心);Sun Yat-sen University(中山大学);Shanghai General Hospital(上海市第一人民医院);Shanghai Jiao Tong University(上海交通大学)
Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
Peking University(北京大学);Peking University Cancer Hospital and Institute(北京大学肿瘤医院暨北京市肿瘤防治研究所);National Engineering Laboratory for Big Data Analysis and Applications, Peking University(北京大学大数据分析与应用技术国家工程实验室);Peking University Changsha Institute for Computing and Digital Economy(北京大学长沙计算与数字经济研究院)
DermETAS-SNA LLM: A Dermatology Focused Evolutionary Transformer Architecture Search with StackNet Augmented LLM Assistant
Data Science Program, Worcester Polytechnic Institute, Massachusetts, USA(沃斯特理工学院数据科学项目,马萨诸塞州,美国);Computer Science Department, Worcester Polytechnic Institute, Massachusetts, USA(沃斯特理工学院计算机科学系,马萨诸塞州,美国);PASE Advisory Group, New Jersey, USA(PASE顾问小组,新泽西州,美国)
PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding
Chongqing University of Posts and Telecommunications(重庆邮电大学);Shanghai First Maternity and Infant Hospital(上海市第一妇婴保健院);School of Medicine Tongji University(同济大学医学院);College of Computer and Information Science, Chongqing Normal University(重庆师范大学计算机与信息科学学院)
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations
School of Computer & Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);School of Computer Science, University of South China(南华大学计算机学院);Faculty of Computer and Information Sciences, Hosei University(法政大学计算机与信息科学学院)
ObGynLongBench: Revealing the Evidence-to-EHR Gap in Longitudinal EHR Decision-Making
School of Data Science, Fudan University(复旦大学数据科学学院);Obstetrics & Gynecology Hospital of Fudan University(复旦大学附属妇产科医院);Shanghai Innovation Institute(上海创新研究院);School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院)
Towards medical AI misalignment: a preliminary study
dept. of Surgical and Medical Science Magna Graecia University of Catanzaro(外科与医学科学系马格拉西亚卡塔尼扎罗大学);dept. of Computer Science Brunel University of London(计算机科学系布鲁内尔伦敦大学);dept. of Computer Engineering, Modelling, Electronics and System University of Calabria(计算机工程、建模、电子与系统系卡塔尼亚大学)
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
NYU Langone Health(纽约大学朗格尼医学中心);The University of Texas at Austin(得克萨斯大学奥斯汀分校);Washington University School of Medicine in St. Louis(圣路易斯华盛顿大学医学院);New York University(纽约大学)
Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
Shenzhen Research Institute of Big Data(大数据研究 institute);National Health Data Institute(国家健康数据研究所);Halmstad University(哈马碧大学);Shenzhen University of Advanced Technology(深圳先进技术大学)
MIRIAD: Augmenting LLMs with millions of medical query-response pairs
Department of Biosystems Science and Engineering, ETH Zurich(生物系统科学与工程系,苏黎世联邦理工学院);Department of Computer Science, Stanford University(计算机科学系,斯坦福大学);Department of Internal Medicine, Mayo Clinic(内科医学系,梅奥诊所);Hugging Face;Department of Radiology, Stanford University(放射学系,斯坦福大学);Hasso-Plattner-Institute for Digital Engineering, University of Potsdam(数字工程研究所,波茨坦大学);Center for Artificial Intelligence in Medicine and Imaging, Stanford, CA, USA(医学与成像人工智能中心,斯坦福,CA,美国);Scripps Translational Science Institute, San Diego, CA, USA(斯克里普斯转化科学研究所,圣地亚哥,CA,美国)
GPT-4 passes most of the 297 written Polish Board Certification Examinations
Adam Mickiewicz University(亚当·密茨凯维奇大学);Poznań University of Medical Sciences(波兹南医科大学);Nicolaus Copernicus University, Collegium Medicum(尼古拉·哥白尼大学医学院);Kazimierz Wielki University(卡齐米日·维尔基大学);The Society for the Substitution Treatment of Addiction “Medically Assisted Recovery”(药物辅助康复成瘾替代治疗协会)
Autonomous Artificial Intelligence Agents for Clinical Decision Making in Oncology
Heidelberg University Hospital(海德堡大学医院);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心);Else Kroener Fresenius Center for Digital Health(Else Kroener Fresenius数字健康中心);Technical University Dresden(德累斯顿工业大学);University of St Andrews(圣安德鲁斯大学);Heidelberg University(海德堡大学);University Hospital RWTH Aachen(亚琛工业大学附属医院);University Hospital Dresden(德累斯顿大学医院);University Medical Center Mainz(美因茨大学医学中心);Technical University of Munich (TUM)(慕尼黑工业大学);Center for Personalized Medicine (ZPM)(个性化医学中心);German Cancer Consortium (DKTK)(德国癌症联盟);Google Cloud(谷歌云);Institute for Cancer Research(癌症研究所);Royal Marsden Hospital(皇家马斯登医院)
OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models
Arizona State University(亚利桑那州立大学);Clemson University(克莱姆森大学);Washington University in St. Louis(圣路易斯华盛顿大学);University of Notre Dame(诺特丹大学);Florida State University(佛罗里达州立大学);Rice University(里德大学);NVIDIA(英伟达);Mayo Clinic(梅奥诊所)