Diagnostic Captioning (DC) concerns the automatic generation of a diagnostic text from a set of medical images of a patient collected during an examination. DC can assist inexperienced physicians, reducing clinical errors. It can also help experienced physicians produce diagnostic reports faster. Following the advances of deep learning, especially in generic image captioning, DC has recently attracted more attention, leading to several systems and datasets. This article is an extensive overview of DC. It presents relevant datasets, evaluation measures, and up to date systems. It also highlights shortcomings that hinder DC's progress and proposes future directions.
Sam-Guided Enhanced Fine-Grained Encoding with Mixed Semantic Learning for Medical Image Captioning
Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University(浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院,浙江大学);Sir Run Run Shaw Hospital, Zhejiang University School of Medicine(浙江大学医学院附属邵逸夫医院)
Regional Attention-Enhanced Swin Transformer for Clinically Relevant Medical Image Captioning
Electrical Engineering and Computer Science(电子工程与计算机科学);Gwangju Institute of Science and Technology(全州科学技术研究院);Electrical and Computer Engineering Department(电子与计算机工程系);Chonnam National University(全南国立大学);AI Convergence Department(人工智能融合部门);Department of Information Systems, CCSIT(信息系统系,CCSIT);King Faisal University(国王费萨尔大学);Thomas Jefferson University, Philadelphia(费城托马斯杰斐逊大学)
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Kimia Lab, Department of Artificial Intelligence \& Informatics, Mayo Clinic, Rochester, MN, USA;Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA;Lane Department of Computer Science;Electrical Engineering, West Virginia University, Morgantown, WV, USA;Department of Computer Science;Engineering, Princeton University, Princeton, NJ, USA;Department of Computer Sciences, University of Wisconsin--Madison, Madison, WI, USA
DermAgent: A Self-Reflective Agentic System for Dermatological Image Analysis with Multi-Tool Reasoning and Traceable Decision-Making
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia(健康人工智能实验室,信息科技学院,墨尔本大学,澳大利亚);Faculty of Information Technology, Monash University, Melbourne, Australia(信息科技学院,墨尔本大学,澳大利亚);University College London, Institute of Ophthalmology, London, United Kingdom(伦敦大学学院,眼科研究所,英国)
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
Hangzhou Normal University(杭州师范大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis
School of Computer Science and Engineering, The Hebrew University of Jerusalem(希伯来大学耶路撒冷分校计算机科学与工程学院);School of Computing and Information Systems, University of Melbourne(墨尔本大学计算与信息学院)
HIMEC: Directional Change Representation and Fixed-Interface Decoding for Remote Sensing Image Change Captioning
University of Electronic Science and Technology of China(电子科技大学);University of Bonn(波恩大学);Bonn-Aachen International Center for Information Technology (B-IT)(波恩-亚琛信息技术国际中心(B-IT))
Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localization
MBZUAI;Provable Responsible AI and Data Analytics (PRADA) Lab(可证明负责任的人工智能与数据分析实验室);King Abdullah University of Science and Technology(卡迪夫大学科学与技术大学);School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉科学学院);University of Copenhagen(哥本哈根大学);Tsinghua University(清华大学)
A survey on deep learning approaches for breast cancer diagnosis
Faculty of Engineering and Applied Science, Ontario Tech University(安大略理工大学工程与应用科学学院);Faculty of Business and Information Technology, Ontario Tech University(安大略理工大学商业与信息技术学院)
A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China(深圳大学计算机科学与软件工程学院);Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, 518055, China(视觉对象检测与识别深圳重点实验室);Laboratory for Artificial Intelligence in Design, Hong Kong(人工智能设计实验室);School of Artificial Intelligence, Shenzhen University, Shenzhen, China(深圳大学人工智能学院)
A Survey of Deep Learning Techniques for the Analysis of COVID-19 and their usability for Detecting Omicron
Pakistan Institute of Engineering & Applied Sciences(巴基斯坦工程与应用科学学院);PIEAS Artificial Intelligence Center (PAIC)(PIEAS人工智能中心);Center for Mathematical Sciences(数学科学中心);University of Engineering and Applied Sciences(工程与应用科学大学);Air University(空军大学)
Visual question answering: from early developments to recent advances -- a survey
Deakin University(迪肯大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of New South Wales(新南威尔士大学);Technology Innovation Institute(技术创新研究院)
Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy
University of Washington(华盛顿大学);New York University(纽约大学);Columbia University(哥伦比亚大学);Liverpool John Moores University(利物浦约翰摩尔大学);Chennai Mathematical Institute(金奈数学研究所);Birla Institute of Technology(比拉理工学院)