通过双流强化学习实现令牌稀疏的医学多模态推理
Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning
AI总结：
提出双流强化学习框架ViToS，通过主动视觉令牌剪枝实现令牌稀疏的医学多模态推理，在七个医学基准上减少77%令牌并提升性能。
摘要：
结合强化学习的视觉语言模型在多模态推理中取得了显著进展，但在医学图像上仍面临挑战，因为医学图像通常包含极其稀疏的视觉证据来支持临床决策。我们认识到，剪除定位区域外的视觉令牌能极大增强医学推理。然而，一个统一的用于主动视觉令牌剪枝和医学多模态推理的强化学习框架尚未建立。为此，我们提出了一个双流强化学习框架ViToS，以实现令牌剪枝和问答。ViToS训练一个具有两个任务分支的策略模型，一个分支专注于定位，另一个分支在视觉令牌剪枝后进行令牌稀疏推理。此外，我们通过引入交叉反馈顺序优化解决了耦合策略学习问题，避免了梯度冲突并促进了共享策略模型的收敛。在七个医学基准上的评估表明，我们的方法将视觉令牌减少到原始序列长度的77%，同时在Lingshu-7B上实现了108.27%的相对性能，在HuatuoGPT-Vision-7B上实现了104.16%的相对性能。总体而言，ViToS提供了优越的性能和推理加速，为医学多模态推理建立了一种高效范式。
英文摘要：
Vision-language models (VLMs) combining reinforcement learning (RL) ignite remarkable progress in multimodal reasoning, yet still struggle with medical images, which typically exhibit extremely sparse visual evidence to inform clinical decision-making. We recognize that pruning visual tokens outside the grounding region greatly enhances medical reasoning. However, a united RL framework for active visual token pruning (VTP) and medical multimodal reasoning remains unestablished. Here, we propose a dual-stream RL framework, ViToS, to fulfill token pruning and question answering. ViToS trains one policy model with two task branches, where one focuses on grounding while the other conducts token-sparse reasoning after VTP. Furthermore, we solve the coupled policy learning problem by introducing the cross-feedback sequential optimization, avoiding gradient conflict and facilitating convergence of the shared policy model. Evaluated on seven medical benchmarks, our method reduces visual tokens to 77% of the original sequence length while achieving a 108.27% relative performance on Lingshu-7B and 104.16% relative performance on HuatuoGPT-Vision-7B. Overall, ViToS delivers superior performance and inference speedup, establishing an efficient paradigm for medical multimodal reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
