BioD2C: A Dual-level Semantic Consistency Constraint Framework for Biomedical VQA
Shandong University(山东大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Institute of Deep Perception Technology, JITRI(JITRI深度感知技术研究院)
Zhengyang Ji, Shang Gao, Li Liu, Yifan Jia, Yutao Yue
Biomedical visual question answering (VQA) has been widely studied and has demonstrated significant application value and potential in fields such as assistive medical diagnosis. Despite their success, current biomedical VQA models perform multimodal information interaction only at the model level within large language models (LLMs), leading to suboptimal multimodal semantic alignment when dealing with complex tasks. To address this issue, we propose BioD2C: a novel Dual-level Semantic Consistency Constraint Framework for Biomedical VQA, which achieves dual-level semantic interaction alignment at both the model and feature levels, enabling the model to adaptively learn visual features based on the question. Specifically, we firstly integrate textual features into visual features via an image-text fusion mechanism as feature-level semantic interaction, obtaining visual features conditioned on the given text; and then introduce a text-queue-based cross-modal soft semantic loss function to further align the image semantics with the question semantics. Specifically, in this work, we establish a new dataset, BioVGQ, to address inherent biases in prior datasets by filtering manually-altered images and aligning question-answer pairs with multimodal context, and train our model on this dataset. Extensive experimental results demonstrate that BioD2C achieves state-of-the-art (SOTA) performance across multiple downstream datasets, showcasing its robustness, generalizability, and potential to advance biomedical VQA research.
MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery(高端工程机械智能制造国家重点实验室);Department of Electrical and Computer Engineering, Sungkyunkwan University(成均馆大学电气与计算机工程系);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院)
RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answering
Raidium;Université de Paris Cité, Hôpital Européen Georges Pompidou, AP-HP(巴黎西岱大学,乔治·蓬皮杜欧洲医院,AP-HP);Department of Vascular and Oncological Interventional Radiology, INSERM(血管与肿瘤介入放射学系,法国国家健康与医学研究院)
Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering
LTCI, Télécom-Paris, Institut Polytechnique de Paris(LTCI, Télécom-Paris, Institut Polytechnique de Paris);Inria, LJK, Univ. Grenoble Alpes(Inria, LJK, 火车头 Grenoble Alpes);Universitat Autónoma de Barcelona(Autonomous University of Barcelona)
Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence
Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);The Hong Kong Polytechnic University(香港理工大学);National University of Singapore(新加坡国立大学);University of Washington(华盛顿大学);Institute of High Performance Computing, Agency for Science, Technology and Research(科技研究局高性能计算研究所)
Biomedical Visual Instruction Tuning with Clinician Preference Alignment
Stanford University(斯坦福大学);Emory University(埃默里大学);University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)
Saliency Guided Longitudinal Medical Visual Question Answering
Dept. of Computer Science and Engineering University of California, San Diego(计算机科学与工程系,加州大学圣地亚哥分校);Dept. of Radiology and Biomedical Imaging Yale University(放射学与生物医学成像系,耶鲁大学)
MedCoT: Medical Chain of Thought via Hierarchical Expert
ZJU-Angelalign R&D Center for Intelligence Healthcare, Zhejiang University(浙江大学ZJU-Angelalign智能医疗研发中心);Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局前沿人工智能研究中心);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所)
Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
Computer Aided Medical Procedures (CAMP)(计算机辅助医疗程序);TU Munich, Germany(慕尼黑工业大学,德国);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Munich, Germany(慕尼黑,德国);Zhongshan Hospital, Fudan University, China(复旦大学中山医院);The University of Hong Kong, Hongkong, China(香港大学,香港,中国)
CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港大学深圳研究院数据科学学院)
Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
TomTom(TomTom公司);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Saarland Informatics Campus(萨尔信息学园区)