医学视觉-语言模型的视觉对齐以实现基于基础的放射学报告生成
Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation
AI总结：
本文提出VALOR方法，通过临床指导文本推理和自监督视觉推理解决医学报告生成中的视觉幻觉问题，提升生成质量与临床准确性。
摘要：
放射学报告生成（RRG）是自动化医疗工作流程的关键步骤，有助于提高患者评估的准确性并减轻医疗专业人员的工作负担。尽管在大型医疗视觉-语言模型（Med-VLMs）方面取得了进展，但生成既视觉基础又临床准确的放射学报告仍是一个重大挑战。现有方法通常依赖于大规模标注语料库、成本高昂的任务特定偏好数据或基于检索的知识。然而，这些策略无法有效缓解由于视觉和语言表示之间对齐不佳导致的幻觉。为了解决这些限制，我们提出了VALOR：医学视觉-语言模型的视觉对齐以实现基于基础的放射学报告生成，通过两个互补的推理阶段解决视觉幻觉问题：（1）临床指导的文本推理通过可验证的自然语言和临床指标奖励引导模型生成具有精确医学术语的语义完整报告。（2）自监督视觉推理利用冻结的领域专家计算输入胸部X光片与生成候选之间的图像-文本相似度分数，并将其转换为排名标准化的优势，明确引导策略向视觉基础输出发展，无需偏好对、检索数据库或额外注释。在多个基准上的广泛实验表明，VALOR显著提高了生成质量和视觉基础的临床准确性，相对于最先进的医疗报告生成基准实现了显著的性能提升。
英文摘要：
Radiology Report Generation (RRG) is a critical step toward automating healthcare workflows, facilitating accurate patient assessments, and reducing the workload of medical professionals. Despite recent progress in Large Medical Vision-Language Models (Med-VLMs), generating radiology reports that are both visually grounded and clinically accurate remains a significant challenge. Existing approaches often rely on large labeled corpora for pre-training, costly task-specific preference data, or retrieval-based knowledge. However, these strategies do not adequately mitigate hallucinations arising from poor cross-modal alignment between visual and linguistic representations. To address these limitations, we propose VALOR: Visual Alignment of Medical Vision-Language Models for GrOunded Radiology Report Generation, which tackles visual hallucinations through two complementary reasoning stages: (1) Clinically Informed Textual Reasoning guides the model with verifiable natural language and clinical metric rewards to produce semantically complete reports with precise medical terminology. (2) Self-Supervised Visual Reasoning leverages a frozen domain expert to compute image-text similarity scores between the input chest X-ray and generated candidates, converting these into rank-normalized advantages that explicitly steer the policy toward visually grounded outputs, requiring no preference pairs, retrieval databases, or additional annotations. Extensive experiments on multiple benchmarks demonstrate that VALOR substantially improves generation quality, as well as clinical accuracy which are visually grounded, achieving significant performance gains over state-of-the-art medical report generation benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
