arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2511.12446cs.CV

CoTBox-TTT:在测试时训练中使用视觉思维链框实现医疗VQA的基础

CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港大学深圳研究院数据科学学院)

Jiahe Qian, Yuhao Shen, Zhangtianyi Chen, Juexiao Zhou, Peisong Wang

AI总结:

针对医疗视觉问答系统在领域转移下失效、答案与图像证据关联弱的问题,提出CoTBox-TTT方法,通过测试时训练更新软提示,利用视觉思维链信号识别相关区域并确保答案一致性,提升模型可靠性与准确率。

摘要:

医疗视觉问答可支持临床决策,但当前系统在领域转移下常失效,生成的答案与图像证据的关联较弱。当模型关注虚假区域,且在部署时重新训练或添加额外标签不可行时,会出现这种可靠性差距。我们用CoTBox-TTT解决这一问题,这是一种证据优先的测试时训练方法,在推理时适应视觉语言模型,同时保持所有骨干网络冻结。该方法仅更新一小部分连续软提示。它通过视觉思维链信号识别问题相关区域,并鼓励原始图像和局部裁剪图像之间的答案一致性。该过程无需标签,可与多种骨干网络即插即用。医疗VQA实验表明,该方法适用于实际部署,例如,在pathVQA上为LLaVA添加CoTBox-TTT可使封闭式准确率提高12.3%。

英文摘要:

Medical visual question answering could support clinical decision making, yet current systems often fail under domain shift and produce answers that are weakly grounded in image evidence. This reliability gap arises when models attend to spurious regions and when retraining or additional labels are impractical at deployment time. We address this setting with CoTBox-TTT, an evidence-first test-time training approach that adapts a vision-language model at inference while keeping all backbones frozen. The method updates only a small set of continuous soft prompts. It identifies question-relevant regions through a visual chain-of-thought signal and encourages answer consistency across the original image and a localized crop. The procedure is label free, and plug and play with diverse backbones. Experiments on medical VQA show that the approach is practical for real deployments. For instance, adding CoTBox-TTT to LLaVA increases closed-ended accuracy by 12.3% on pathVQA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑