通过目光思考：将眼动作为视觉推理监督用于医学视觉语言模型
Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs
AI总结：
通过引入眼动标记，利用时间有序的注视轨迹引导医学VLM的视觉推理，提升放射学任务的性能与鲁棒性。
摘要：
视觉语言模型（VLMs）将图像处理为视觉标记，但其中间推理通常在文本中进行，这在需要视觉支持的放射学任务中可能不够优化。放射学家则通过连续的视觉搜索进行诊断；眼动捕捉这一过程作为时间有序的注视轨迹，揭示了证据随时间的获取方式。我们使用眼动作为监督，通过引入少量专门的眼动标记来引导VLM的推理。这些标记被训练以按时间顺序预测被注视的图像块索引，促使模型遵循人类般的证据获取和整合过程。在MIMIC-EYE和多个外部零样本基准上的实验表明，与基线相比有持续的提升，实现了领域内最先进的性能，并改善了领域外的鲁棒性。这些结果突显了时间有序的注视作为学习视觉支持的医学推理的有效监督信号。
英文摘要：
Vision--language models (VLMs) process images as visual tokens, yet their intermediate reasoning is often carried out in text, which can be suboptimal for visually grounded radiology tasks. Radiologists instead diagnose via sequential visual search; eye-tracking captures this process as time-ordered gaze trajectories that reveal how evidence is acquired over time. We use eye-gaze as supervision to guide VLM reasoning by introducing a small set of dedicated gaze tokens. These tokens are trained to predict gaze-selected image patch indices in temporal order, encouraging the model to follow human-like evidence acquisition and integration. Experiments on MIMIC-EYE and multiple external zero-shot benchmarks show consistent gains over baselines, achieving state-of-the-art in-domain performance and improved out-of-domain robustness. These results highlight temporally ordered gaze as an effective supervision signal for learning visually grounded medical reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
