CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation
Academy for Engineering and Technology, Fudan University(复旦大学工程技术研究院);Tencent YouTu Lab(腾讯优图实验室);Xiamen University(厦门大学);Institute of Meta-Medical, Fudan University(复旦大学元医学研究院);Engineering Research Center of AI and Robotics, Ministry of Education(教育部人工智能与机器人工程研究中心);Jilin Provincial Key Laboratory of Intelligence Science and Engineering(吉林省智能科学与工程重点实验室)
Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. The code and dataset have been released at https://github.com/FRENKIE-CHIANG/CoMT.
CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models
School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院);Key Laboratory of Data Intelligence and Advanced Computing in Provincial Universities, Soochow University(苏州大学省级高校数据智能与先进计算重点实验室);Research Center for SCIR, Harbin Institute of Technology(哈尔滨工业大学SCIR研究中心);National University of Singapore(新加坡国立大学)
Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models
School of Computer Science and Ningbo Institute, Northwestern Polytechnical University, China(西北工业大学计算机学院和宁波研究院);National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, China(国家空天地海一体化大数据应用技术国家工程实验室)
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Johns Hopkins University, School of Medicine(约翰霍普金斯大学医学院);Massachusetts Institute of Technology(麻省理工学院);Microsoft Healthcare & Life Sciences(微软医疗健康与生命科学)
CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Ministry of Education Key Laboratory of Intelligent Networks and Network Security, China(教育部智能网络与网络安全重点实验室);Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, China(陕西省大数据知识工程重点实验室);IHPC, Agency for Science, Technology and Research, Singapore(新加坡科技研究局IHPC);Show Lab, National University of Singapore(新加坡国立大学Show实验室);College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院)
TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis
Zhejiang University(浙江大学);DAMO Academy, Alibaba Group(阿里集团达摩院);Hupan Lab(虎扑实验室);Shanghai Institution of Pancreatic Disease(上海胰腺疾病研究所);Shengjing Hospital of China Medical University(中国医科大学盛京医院);Sun Yat-sen University Cancer Center(中山大学肿瘤中心)
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Harbin Institute of Technology, Shenzhen(深圳哈尔滨工业大学);University of Science and Technology Beijing(北京科技大学)
Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(厦门大学 多媒体可信感知与高效计算重点实验室(教育部));Tencent Youtu Lab(腾讯优图实验室)
CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
Center for Artificial Intelligence in Medicine and Imaging(医学与影像人工智能中心);Department of Radiology(放射科);Department of Pediatrics(儿科);Department of Radiology, Weill Cornell Medicine(韦氏 Cornell 医学院放射科);Department of Radiology, Brigham and Women’s Hospital(布里奇妇产科医院放射科);University of California, Berkeley(加州大学伯克利分校);University of California, San Francisco(加州大学旧金山分校);Department of Biomedical Data Science(生物医学数据科学部)
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving
School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动学院);McGill University(麦吉尔大学);Automotive and Robotics, Xiaomi Corporation(小米公司汽车与机器人部门);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)