VisionFM：面向通用眼科人工智能的多模态多任务视觉基础模型
VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence
AI总结：
VisionFM是一个基于340万张眼科图像预训练的多模态多任务基础模型，在12种常见眼科疾病诊断中超越中级眼科医生，并展现出强泛化能力，为眼科AI提供高效基础。
摘要：
我们提出了VisionFM，这是一个在来自560,457名个体的340万张眼科图像上预训练的基础模型，涵盖了广泛的眼科疾病、模态、成像设备和人口统计学特征。预训练后，VisionFM为促进多种眼科人工智能（AI）应用提供了基础，如疾病筛查与诊断、疾病预后、疾病表型亚型分类以及系统性生物标志物和疾病预测，每项应用均具备专家级智能和准确性。VisionFM的通用智能在联合诊断12种常见眼科疾病方面优于具有基础和中等级别的眼科医生。在一个新的大规模眼科疾病诊断基准数据库以及一个新的大规模分割和检测基准数据库上评估时，VisionFM优于强基线深度神经网络。VisionFM学习的眼科图像表示表现出显著的可解释性，并展示了对新眼科模态、疾病谱和成像设备的强大泛化能力。作为一个基础模型，VisionFM具有从多样眼科成像数据和不同数据集中学习的大容量。为了与此容量相匹配，除了用于预训练的真实数据外，我们还生成并利用了合成眼科成像数据。实验结果表明，通过视觉图灵测试的合成数据也能增强VisionFM的表示学习能力，从而在下游眼科AI任务中带来显著的性能提升。除了本工作中开发、验证和展示的眼科AI应用外，使用VisionFM作为基础，还可以以高效且经济的方式实现大量进一步的应用。
英文摘要：
We present VisionFM, a foundation model pre-trained with 3.4 million ophthalmic images from 560,457 individuals, covering a broad range of ophthalmic diseases, modalities, imaging devices, and demography. After pre-training, VisionFM provides a foundation to foster multiple ophthalmic artificial intelligence (AI) applications, such as disease screening and diagnosis, disease prognosis, subclassification of disease phenotype, and systemic biomarker and disease prediction, with each application enhanced with expert-level intelligence and accuracy. The generalist intelligence of VisionFM outperformed ophthalmologists with basic and intermediate levels in jointly diagnosing 12 common ophthalmic diseases. Evaluated on a new large-scale ophthalmic disease diagnosis benchmark database, as well as a new large-scale segmentation and detection benchmark database, VisionFM outperformed strong baseline deep neural networks. The ophthalmic image representations learned by VisionFM exhibited noteworthy explainability, and demonstrated strong generalizability to new ophthalmic modalities, disease spectrum, and imaging devices. As a foundation model, VisionFM has a large capacity to learn from diverse ophthalmic imaging data and disparate datasets. To be commensurate with this capacity, in addition to the real data used for pre-training, we also generated and leveraged synthetic ophthalmic imaging data. Experimental results revealed that synthetic data that passed visual Turing tests, can also enhance the representation learning capability of VisionFM, leading to substantial performance gains on downstream ophthalmic AI tasks. Beyond the ophthalmic AI applications developed, validated, and demonstrated in this work, substantial further applications can be achieved in an efficient and cost-effective manner using VisionFM as the foundation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
