CLARIFY：用于准确且轻量级皮肤病学视觉问答的专家-通用框架
CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering
AI总结：
针对视觉语言模型在皮肤病学视觉问答中专业诊断准确性不足和推理成本高的问题，提出CLARIFY专家-通用框架，结合轻量领域图像分类器与压缩对话式VLM，提升诊断准确性并降低计算成本。
摘要：
视觉语言模型（VLMs）在医疗任务中展现出显著潜力；然而，其通用性质可能限制专业诊断准确性，且庞大尺寸给真实临床部署带来高昂推理成本。为解决这些挑战，我们引入CLARIFY，一种用于皮肤病学视觉问答（VQA）的专家-通用框架。CLARIFY包含两个组件：（i）轻量级、领域训练的图像分类器（专家），提供快速且高精度的诊断预测；（ii）强大但压缩的对话式VLM（通用），生成自然语言解释以响应用户查询。在我们的框架中，专家的预测直接引导通用的推理，使其聚焦正确诊断路径。这种协同作用通过基于知识图谱的检索模块进一步增强，该模块将通用的响应基于事实皮肤病学知识，确保准确性和可靠性。这种分层设计不仅减少诊断错误，还显著提高计算效率。在我们精心构建的多模态皮肤病数据集上的实验表明，CLARIFY相比最强基线（微调、未压缩的单一线性VLM）诊断准确性提升18%，同时平均VRAM需求和延迟分别至少降低20%和5%。这些结果表明，专家-通用系统为构建轻量级、可信且临床可行的AI系统提供了实用且强大的范式。
英文摘要：
Vision-language models (VLMs) have shown significant potential for medical tasks; however, their general-purpose nature can limit specialized diagnostic accuracy, and their large size poses substantial inference costs for real-world clinical deployment. To address these challenges, we introduce CLARIFY, a Specialist-Generalist framework for dermatological visual question answering (VQA). CLARIFY combines two components: (i) a lightweight, domain-trained image classifier (the Specialist) that provides fast and highly accurate diagnostic predictions, and (ii) a powerful yet compressed conversational VLM (the Generalist) that generates natural language explanations to user queries. In our framework, the Specialist's predictions directly guide the Generalist's reasoning, focusing it on the correct diagnostic path. This synergy is further enhanced by a knowledge graph-based retrieval module, which grounds the Generalist's responses in factual dermatological knowledge, ensuring both accuracy and reliability. This hierarchical design not only reduces diagnostic errors but also significantly improves computational efficiency. Experiments on our curated multimodal dermatology dataset demonstrate that CLARIFY achieves an 18\% improvement in diagnostic accuracy over the strongest baseline, a fine-tuned, uncompressed single-line VLM, while reducing the average VRAM requirement and latency by at least 20\% and 5\%, respectively. These results indicate that a Specialist-Generalist system provides a practical and powerful paradigm for building lightweight, trustworthy, and clinically viable AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
