arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2611.22652cs.CV
收藏

OmniTongue:面向中医舌诊的多模态大语言模型综合基准

OmniTongue: A Comprehensive MLLM Benchmark for Automated Tongue Diagnosis in Traditional Chinese Medicine

College of Information Science and Engineering, Northeastern University(东北大学信息科学与工程学院);College of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院)

Liuyi Xu, Yun Guo, Shusheng Li, Xuewen Zhang, Yining Qian, An-Yang Lu

AI总结:

研究针对现有中医舌诊数据集的局限,推出OmniTongue综合基准,评估发现现有MLLM在舌诊中性能差距显著,凸显该基准对开发自动化舌诊模型的重要性。

摘要:

多模态大语言模型(MLLM)在西医医学影像领域已表现出色,但其在中医(TCM)舌诊中的潜力尚未得到系统评估。舌诊需要复杂的全局与局部感知,是评估MLLM视觉理解深度的严格试金石。然而,现有数据集存在维度单一、粒度粗糙、标签模糊的局限,阻碍了对真实临床能力的评估。为填补这一空白,我们推出OmniTongue,首个面向中医舌诊的综合细粒度视觉问答基准。该基准整合了1351张经专家验证的图像与21616条指令对,核心设计包含16个临床维度用于多维度评估、针对病理定位的区域特定标注,以及66个细粒度标签用于精准诊断表征。对现有MLLM的广泛评估显示出显著的性能差距,表明医学专用模型缺乏领域优势,而普遍较低的Macro-F1分数则暴露了模型依赖统计捷径而非真正理解舌病理的问题。这些发现不仅凸显了MLLM在解读区域医学特征方面的当前局限,也强调了OmniTongue对开发稳健自动化舌诊模型的重要性。代码与数据集将在后续版本中公开。

英文摘要:

Multimodal Large Language Models (MLLMs) have excelled in Western medical imaging, yet their potential in Traditional Chinese Medicine (TCM) tongue diagnosis remains systematically unevaluated. Tongue diagnosis requires sophisticated global and local perception, serving as a rigorous touchstone for assessing the visual understanding depth of MLLMs. However, existing datasets are limited by single dimensions, coarse granularity, and ambiguous labels, hindering the evaluation of authentic clinical capabilities. To bridge this gap, we introduce OmniTongue, the first comprehensive, fine-grained Visual Question Answering benchmark for TCM tongue diagnosis. This benchmark integrates 1,351 expert-validated images and 21,616 instruction pairs. Its core design features 16 clinical dimensions for multidimensional evaluation, region-specific annotations for pathological localization, and 66 fine-grained labels for precise diagnostic representation. Extensive evaluations of existing MLLMs reveal significant performance gaps, demonstrating that medical-specific models lack domain advantages, while universally poor Macro-F1 scores expose a reliance on statistical shortcuts rather than genuine understanding of tongue pathology. These findings not only highlight the current limitations of MLLMs in interpreting regional medical features but also underscore OmniTongue's significance for developing robust automated tongue diagnosis models. The code and dataset will be made publicly available in a future release.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑