arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.14243cs.CLcs.AIcs.CV
收藏

Dermacen Analytica:一种将多模态大语言模型与机器学习相结合的新型远程皮肤病学方法论

Dermacen Analytica: A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatology

University of Piraeus(比雷埃夫斯大学);Athens University Medical School(雅典医学院)

Dimitrios P. Panagoulias, Evridiki Tsoureli-Nikita, Maria Virvou, George A. Tsihrintzis

AI总结:

本文提出一种整合多模态大语言模型与机器学习的新型远程皮肤病学方法论,通过跨模型验证和专家评估,在上下文理解和诊断准确性上达到约0.87的加权分数,旨在提升皮肤病诊断及远程医疗服务。

摘要:

人工智能的兴起在医学发现、诊断和患者管理领域带来了巨大的希望。然而,所有医学领域的巨大复杂性需要一种更复杂的方法,将机器学习算法、分类器、分割算法以及最近的大语言模型相结合。在本文中,我们描述、实施并评估了一个由人工智能驱动的系统和 methodology,旨在协助皮肤病学领域内皮肤病变和其他皮肤状况的诊断过程,该 methodology 旨在全面解决该领域的诊断过程。该工作流程整合了大语言模型、基于 transformer 的视觉模型和复杂的机器学习工具。这种整体方法实现了对皮肤病状况的细致解读,模拟并促进了皮肤科医生的工作流程。我们通过一种嵌入在评估流程中的全面跨模型验证技术来评估我们提出的 methodology,该评估流程利用公开可用的皮肤病状况医学案例研究和相关图像。为了定量评分系统性能,我们采用了先进的机器学习和自然语言处理工具,这些工具侧重于相似性比较和自然语言推理。此外,我们结合了一个基于结构化检查表的人类专家评估过程,以进一步验证我们的结果。我们在一个系统中实施了所提出的 methodology,该系统在上下文理解和诊断准确性方面均获得了约0.87的(加权)分数,证明了我们的方法在增强皮肤病学分析方面的有效性。所提出的 methodology 预计将在下一代远程皮肤病学应用的开发中发挥作用,增强远程咨询能力和护理可及性,尤其是在服务不足的地区。

英文摘要:

The rise of Artificial Intelligence creates great promise in the field of medical discovery, diagnostics and patient management. However, the vast complexity of all medical domains require a more complex approach that combines machine learning algorithms, classifiers, segmentation algorithms and, lately, large language models. In this paper, we describe, implement and assess an Artificial Intelligence-empowered system and methodology aimed at assisting the diagnosis process of skin lesions and other skin conditions within the field of dermatology that aims to holistically address the diagnostic process in this domain. The workflow integrates large language, transformer-based vision models and sophisticated machine learning tools. This holistic approach achieves a nuanced interpretation of dermatological conditions that simulates and facilitates a dermatologist's workflow. We assess our proposed methodology through a thorough cross-model validation technique embedded in an evaluation pipeline that utilizes publicly available medical case studies of skin conditions and relevant images. To quantitatively score the system performance, advanced machine learning and natural language processing tools are employed which focus on similarity comparison and natural language inference. Additionally, we incorporate a human expert evaluation process based on a structured checklist to further validate our results. We implemented the proposed methodology in a system which achieved approximate (weighted) scores of 0.87 for both contextual understanding and diagnostic accuracy, demonstrating the efficacy of our approach in enhancing dermatological analysis. The proposed methodology is expected to prove useful in the development of next-generation tele-dermatology applications, enhancing remote consultation capabilities and access to care, especially in underserved areas.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑