Multimodal large language models are promising for clinical visual question answering tasks, but scaling to 3D imaging is hindered by high computational costs. Prior methods often rely on 2D slices or fixed-length token compression, disrupting volumetric continuity and obscuring subtle findings. We present Photon, a framework that represents 3D medical volumes with token sequences of variable length. Photon introduces instruction-conditioned token scheduling and surrogate gradient propagation to adaptively reduce tokens during both training and inference, which lowers computational cost while mitigating the attention dilution caused by redundant tokens. It incorporates a custom backpropagation rule with gradient restoration to enable differentiable optimization despite discrete token drop. To stabilize token compression and ensure reliable use of visual evidence, Photon further applies regularization objectives that mitigate language-only bias and improve reliability. Experiments on diverse medical visual question answering tasks show that Photon achieves state-of-the-art accuracy while reducing resource usage and accelerating both training and inference.
Volumetric Radiology AI in the Era of Multimodal Large Language Models
Fudan University（复旦大学）；Northwestern Polytechnical University（西北工业大学）；Xi’an Jiaotong University（西安交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Southern Medical University（南方医科大学）；The Chinese University of Hong Kong（香港中文大学）；University of British Columbia（不列颠哥伦比亚大学）；Microsoft Research（微软研究院）；Westlake University（西湖大学）
BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning
University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；University of Pittsburgh（匹兹堡大学）；The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；University of Dublin, Trinity College（都柏林大学圣三一学院）；Yale University（耶鲁大学）
Speculative Decoding Reimagined for Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室、中华人民共和国教育部、厦门大学）
LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences（智能机器研究所，合肥物理科学研究院，中国科学院）；University of Science and Technology of China（中国科学技术大学）；Zhejiang University（浙江大学）；East China Normal University（华东师范大学）；The Hong Kong University of Science and Technology（香港科技大学）
MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation
Department of Biomedical Engineering, University of Florida（佛罗里达大学生物医学工程系）；Department of Radiology, University of Florida（佛罗里达大学放射学系）；Research Computing, University of Florida（佛罗里达大学研究计算中心）；Department of Medicine, University of Florida（佛罗里达大学医学系）；Department of Radiology, UC San Francisco（旧金山大学放射学系）
École Polytechnique Fédérale de Lausanne (EPFL), Switzerland（瑞士联邦理工学院洛桑校区）；ETH Zürich, Switzerland（瑞士苏黎世联邦理工学院）；T.H. Chan School of Public Health, Harvard University, USA（哈佛大学T.H. Chan公共卫生学院）
MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
University of Michigan（密歇根大学）；The Ohio State University（俄亥俄州立大学）；Independent（独立）；Indiana University（印第安纳大学）；The University of Hong Kong（香港大学）；Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）；LMSYS Org（LMSYS组织）
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Harbin Institute of Technology, Shenzhen（深圳哈尔滨工业大学）；University of Science and Technology Beijing（北京科技大学）
MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs
Arizona State University（亚利桑那州立大学）；University of Washington（华盛顿大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Duke University（杜克大学）；Zoom Communications（Zoom通讯）
Massachusetts General Hospital（麻省总医院）；Massachusetts Institute of Technology（麻省理工学院）；National Taiwan University Hospital（台湾大学医院）；Vysioneer Inc.（Vysioneer公司）
Stereotactic radiosurgery (SRS), which delivers high doses of irradiation in a single or few shots to small targets, has been a standard of care for brain metastases. While very effective, SRS currently requires manually intensive delineation of tumors. In this work, we present a deep learning approach for automated detection and segmentation of brain metastases using multimodal imaging and ensemble neural networks. In order to address small and multiple brain metastases, we further propose a volume-aware Dice loss which optimizes model performance using the information of lesion size. This wo…
Multimodal Slice Interaction Network Enhanced by Transfer Learning for Precise Segmentation of Internal Gross Tumor Volume in Lung Cancer PET/CT Imaging
Dept. of Biomedical Engineering Johns Hopkins University（生物医学工程系约翰霍普金斯大学）；Molecular Radiation Sciences Johns Hopkins University（分子放射科学约翰霍普金斯大学）；Dept. of Surgery University of Minnesota（外科系明尼苏达大学）；Dept. of Biomedical Engineering University of Virginia（生物医学工程系弗吉尼亚大学）；Comprehensive Cancer Center Mayo Clinic（综合癌症中心梅奥诊所）
LLM-driven Multimodal Target Volume Contouring in Radiation Oncology
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Yonsei University College of Medicine（延世大学医学院）；Yonsei University（延世大学）；Yongin Severance Hospital（龙仁世福兰斯医院）；Gangnam Severance Hospital（江南世福兰斯医院）；Oncosoft Inc.（Oncosoft公司）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Chinese PLA General Hospital（中国人民解放军总医院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；Beijing Jiaotong University（北京交通大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
In neuroimaging analysis, fMRI can well assess the function changes for brain diseases with no obvious structural lesions. To date, most deep-learning-based fMRI studies have employed functional connectivity (FC) as the basic feature for disease classification. However, FC is calculated on time series of predefined regions of interest and neglects detailed information contained in each voxel. Another drawback of using FC is the limited sample size for the training of deep models. The low representation ability of FC leads to poor performance in clinical practice, especially when dealing with m…
University of Florida（佛罗里达大学）；Medical Imaging Technologies, Siemens Healthcare（西门子医疗医学影像技术部门）
Synthesized medical images have several important applications, e.g., as an intermedium in cross-modality image registration and as supplementary training samples to boost the generalization capability of a classifier. Especially, synthesized computed tomography (CT) data can provide X-ray attenuation map for radiation therapy planning. In this work, we propose a generic cross-modality synthesis approach with the following targets: 1) synthesizing realistic looking 3D images using unpaired training data, 2) ensuring consistent anatomical structures, which could be changed by geometric distorti…
Head movement during scanning impedes activation detection in fMRI studies. Head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) can be estimated and compensated by aligning the images onto a reference volume through image registration. However, registering EPI images volume to volume fails to consider head motion between slices, which may lead to severely biased head motion estimates. Slice-to-volume registration can be used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, accurate slice to volume mapping…
