基于视觉语言模型提示的无配对组织病理学虚拟染色的最优辅助方法
VLM-based Prompts as the Optimal Assistant for Unpaired Histopathology Virtual Staining
AI总结：
本研究首次将病理视觉语言大模型作为辅助工具，通过对比可学习提示和概念锚点，结合VLM约束的数据增强，实现无配对组织病理学虚拟染色，提升图像真实性和下游诊断精度。
摘要：
在组织病理学中，组织切片通常使用常见的H&E染色或特殊染色（如MAS、PAS、PASM等）来清晰显示特定的组织结构。深度学习的快速发展为生成虚拟染色图像提供了有效解决方案，显著降低了传统组织化学染色所需的时间和人力成本。然而，一个新的挑战出现了：如何将组织切片的基本视觉特征与染色剂引起的视觉差异分离开来。此外，虚拟染色常常忽略必要的病理知识和染色的物理特性，导致仅实现风格层面的转换。为解决这些问题，我们首次在虚拟染色任务中引入病理视觉语言大模型（VLM）作为辅助工具。我们整合了对比可学习提示、组织切片的基础概念锚点以及染色特异性概念锚点，以利用病理VLM的广泛知识。该方法旨在描述、框定并增强虚拟染色的方向。此外，我们开发了一种基于VLM约束的数据增强方法。该方法利用VLM强大的图像解释能力，进一步整合图像风格和结构信息，在高精度病理诊断中被证明是有益的。在公开可用的多域无配对染色数据集上进行的大量评估表明，我们的方法能够生成高度逼真的图像，并提高下游任务（如肾小球检测和分割）的准确性。我们的代码可在以下网址获取：https://github.com/CZZZZZZZZZZZZZZZZZ/VPGAN-HARBOR
英文摘要：
In histopathology, tissue sections are typically stained using common H&E staining or special stains (MAS, PAS, PASM, etc.) to clearly visualize specific tissue structures. The rapid advancement of deep learning offers an effective solution for generating virtually stained images, significantly reducing the time and labor costs associated with traditional histochemical staining. However, a new challenge arises in separating the fundamental visual characteristics of tissue sections from the visual differences induced by staining agents. Additionally, virtual staining often overlooks essential pathological knowledge and the physical properties of staining, resulting in only style-level transfer. To address these issues, we introduce, for the first time in virtual staining tasks, a pathological vision-language large model (VLM) as an auxiliary tool. We integrate contrastive learnable prompts, foundational concept anchors for tissue sections, and staining-specific concept anchors to leverage the extensive knowledge of the pathological VLM. This approach is designed to describe, frame, and enhance the direction of virtual staining. Furthermore, we have developed a data augmentation method based on the constraints of the VLM. This method utilizes the VLM's powerful image interpretation capabilities to further integrate image style and structural information, proving beneficial in high-precision pathological diagnostics. Extensive evaluations on publicly available multi-domain unpaired staining datasets demonstrate that our method can generate highly realistic images and enhance the accuracy of downstream tasks, such as glomerular detection and segmentation. Our code is available at: https://github.com/CZZZZZZZZZZZZZZZZZ/VPGAN-HARBOR
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
