LLaDA-MedV：探索大规模语言扩散模型用于生物医学图像理解
LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding
AI总结：
LLaDA-MedV通过视觉指令微调，首次在生物医学图像理解中应用大规模语言扩散模型，实现了在生物医学视觉对话任务中的性能提升，并在多个VQA基准测试中取得新高准确率。
摘要：
自回归模型（ARMs）长期以来在生物医学视觉语言模型（VLMs）领域占据主导地位。最近，掩码扩散模型（如LLaDA）作为有前景的替代方案出现，但其在生物医学领域的应用仍鲜有探索。为弥补这一空白，我们引入LLaDA-MedV，这是首个专门针对生物医学图像理解的大型语言扩散模型，通过视觉指令微调实现。LLaDA-MedV在开放式的生物医学视觉对话任务中相对于LLaVA-Med提升了7.855%，相对于LLaDA-V提升了1.867%。并且在三个VQA基准测试的闭合形式子集上设定了新的最先进准确率：在VQA-RAD上为84.93%，在SLAKE上为92.31%，在PathVQA上为95.15%。此外，与LLaVA-Med的详细比较表明，LLaDA-MedV能够通过显式控制响应长度生成合理更长的响应，这可以导致更有信息的输出。我们还深入分析了训练和推理阶段，强调了初始化权重选择、微调策略以及采样步骤与响应重复之间的相互作用的重要性。代码和模型权重已发布在https://github.com/LLM-VLM-GSL/LLaDA-MedV。
英文摘要：
Autoregressive models (ARMs) have long dominated the landscape of biomedical vision-language models (VLMs). Recently, masked diffusion models such as LLaDA have emerged as promising alternatives, yet their application in the biomedical domain remains largely underexplored. To bridge this gap, we introduce LLaDA-MedV, the first large language diffusion model tailored for biomedical image understanding through vision instruction tuning. LLaDA-MedV achieves relative performance gains of 7.855% over LLaVA-Med and 1.867% over LLaDA-V in the open-ended biomedical visual conversation task, and sets new state-of-the-art accuracy on the closed-form subset of three VQA benchmarks: 84.93% on VQA-RAD, 92.31% on SLAKE, and 95.15% on PathVQA. Furthermore, a detailed comparison with LLaVA-Med suggests that LLaDA-MedV is capable of generating reasonably longer responses by explicitly controlling response length, which can lead to more informative outputs. We also conduct an in-depth analysis of both the training and inference stages, highlighting the critical roles of initialization weight selection, fine-tuning strategies, and the interplay between sampling steps and response repetition. The code and model weight is released at https://github.com/LLM-VLM-GSL/LLaDA-MedV.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
