推进慢性结核病诊断的视觉-语言模型：用于精准分析的多模态框架
Advancing Chronic Tuberculosis Diagnostics Using Vision-Language Models: A Multi modal Framework for Precision Analysis
AI总结：
本研究提出一种结合SIGLIP编码器和Gemma-3b解码器的视觉-语言模型，融合胸部X光与临床数据，实现慢性结核病高精度自动诊断，精度和召回率达94%。
摘要：
背景：本研究提出了一种视觉-语言模型（VLM），利用SIGLIP编码器和Gemma-3b变压器解码器来增强自动化慢性结核病（TB）筛查。通过将胸部X光图像与临床数据相结合，该模型解决了手动解读的挑战，提高了诊断的一致性和可及性，尤其是在资源受限的环境中。方法：VLM架构结合了用于视觉编码的视觉变压器（ViT）和用于处理临床背景（如患者病史和治疗记录）的基于变压器的文本编码器。跨模态注意力机制将放射学特征与文本信息对齐，而Gemma-3b解码器生成全面的诊断报告。该模型在500万对医学图像和文本上进行了预训练，并使用10万张慢性结核病特异性胸部X光片进行了微调。结果：该模型在检测关键慢性结核病病理（包括纤维化、钙化肉芽肿和支气管扩张）方面表现出高精度（94%）和高召回率（94%）。曲线下面积（AUC）得分超过0.93，交并比（IoU）值高于0.91，验证了其在检测和定位结核病相关异常方面的有效性。结论：VLM为自动化慢性结核病诊断提供了一种稳健且可扩展的解决方案，整合了放射学和临床数据，以提供可操作且具有上下文意识的见解。未来的工作将解决细微病理和数据集偏差问题，以增强模型的泛化能力，确保在不同人群和医疗环境中的公平性能。
英文摘要：
Background: This study proposes a Vision-Language Model (VLM) leveraging the SIGLIP encoder and Gemma-3b transformer decoder to enhance automated chronic tuberculosis (TB) screening. By integrating chest X-ray images with clinical data, the model addresses the challenges of manual interpretation, improving diagnostic consistency and accessibility, particularly in resource-constrained settings. Methods: The VLM architecture combines a Vision Transformer (ViT) for visual encoding and a transformer-based text encoder to process clinical context, such as patient histories and treatment records. Cross-modal attention mechanisms align radiographic features with textual information, while the Gemma-3b decoder generates comprehensive diagnostic reports. The model was pre-trained on 5 million paired medical images and texts and fine-tuned using 100,000 chronic TB-specific chest X-rays. Results: The model demonstrated high precision (94 percent) and recall (94 percent) for detecting key chronic TB pathologies, including fibrosis, calcified granulomas, and bronchiectasis. Area Under the Curve (AUC) scores exceeded 0.93, and Intersection over Union (IoU) values were above 0.91, validating its effectiveness in detecting and localizing TB-related abnormalities. Conclusion: The VLM offers a robust and scalable solution for automated chronic TB diagnosis, integrating radiographic and clinical data to deliver actionable and context-aware insights. Future work will address subtle pathologies and dataset biases to enhance the model's generalizability, ensuring equitable performance across diverse populations and healthcare settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
