多模态大语言模型在胃肠诊断中的临床认知对齐
Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs
AI总结：
本文提出CogAlign框架，通过构建层次化临床认知数据集和监督微调提升模型临床分析能力，并采用反事实强化学习消除视觉偏差，实现胃肠诊断的因果关联与高准确率。
摘要：
多模态大语言模型（MLLMs）在医学图像分析中展现出巨大潜力。然而，其在胃肠内镜中的应用受限于两个关键问题：通用模型推理与标准化临床认知路径的不匹配，以及视觉特征与诊断结果之间的因果关联缺失。本文提出临床-认知对齐（CogAlign）框架，首先通过构建层次化临床认知数据集和监督微调赋予模型严谨的临床分析能力，内部化专家的层次化诊断逻辑，从解剖定位、形态评估到微血管分析。其次，为消除视觉偏差，通过理论分析证明标准监督微调不可避免收敛到虚假背景相关性，并提出反事实驱动的强化学习策略，通过病变遮蔽生成反事实正常样本并优化临床-认知中心奖励，使模型严格基于因果病变特征进行诊断。大量实验表明，本文方法在多个基准上均达到最先进水平，显著提升复杂临床场景下的诊断准确率。所有源代码和数据集将公开发布。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis. However, their application in gastrointestinal endoscopy is currently hindered by two critical limitations: the misalignment between general model reasoning and standardized clinical cognitive pathways, and the lack of causal association between visual features and diagnostic outcomes. In this paper, we propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges. First, we endow the model with rigorous clinical analytical capabilities by constructing the hierarchical clinical cognition dataset and employing Supervised Fine-Tuning (SFT). Unlike conventional approaches, this strategy internalizes the hierarchical diagnostic logic of experts, ranging from anatomical localization and morphological evaluation to microvascular analysis, directly into the model. Second, to eliminate visual bias, we provide a theoretical analysis demonstrating that standard supervised tuning inevitably converges to spurious background correlations. Guided by this insight, we propose a counterfactual-driven reinforcement learning strategy to enforce causal rectification. By generating counterfactual normal samples via lesion masking and optimizing through clinical-cognition-centric rewards, we constrain the model to strictly ground its diagnosis in causal lesion features. Extensive experiments demonstrate that our approach achieves State-of-the-Art (SoTA) performance across multiple benchmarks, significantly enhancing diagnostic accuracy in complex clinical scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
