CheXPO：基于反事实推理的胸部X光视觉语言模型偏好优化
CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationale
AI总结：
针对胸部X光视觉语言模型的幻觉问题，提出结合置信度-相似性挖掘与反事实推理的偏好优化策略CheXPO，仅用5%样本实现8.93%性能提升，达到最先进水平。
摘要：
视觉语言模型（VLM）容易产生幻觉，这严重损害了其在医疗应用中的可靠性。虽然偏好优化可以通过临床反馈减轻这些幻觉，但其实现面临诸多挑战，如临床无关的训练样本、不平衡的数据分布以及高昂的专家标注成本。为解决这些挑战，我们引入了CheXPO，一种胸部X光偏好优化策略，该策略将置信度-相似性联合挖掘与反事实推理相结合。我们的方法首先针对不同问题类型合成一个统一、细粒度的多任务胸部X光视觉指令数据集，用于监督微调（SFT）。随后，我们通过对SFT失败样本的令牌级置信度分析来识别困难样本，并利用基于相似性的检索来扩展困难样本以平衡偏好样本分布，同时合成反事实推理提供细粒度的临床偏好，从而无需额外专家输入。实验表明，CheXPO仅使用5%的SFT样本便实现了8.93%的相对性能提升，在多种临床任务中达到最先进水平，为真实世界放射学应用提供了可扩展、可解释的解决方案。
英文摘要：
Vision-language models (VLMs) are prone to hallucinations that critically compromise reliability in medical applications. While preference optimization can mitigate these hallucinations through clinical feedback, its implementation faces challenges such as clinically irrelevant training samples, imbalanced data distributions, and prohibitive expert annotation costs. To address these challenges, we introduce CheXPO, a Chest X-ray Preference Optimization strategy that combines confidence-similarity joint mining with counterfactual rationale. Our approach begins by synthesizing a unified, fine-grained multi-task chest X-ray visual instruction dataset across different question types for supervised fine-tuning (SFT). We then identify hard examples through token-level confidence analysis of SFT failures and use similarity-based retrieval to expand hard examples for balancing preference sample distributions, while synthetic counterfactual rationales provide fine-grained clinical preferences, eliminating the need for additional expert input. Experiments show that CheXPO achieves 8.93% relative performance gain using only 5% of SFT samples, reaching state-of-the-art performance across diverse clinical tasks and providing a scalable, interpretable solution for real-world radiology applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
