基于跨架构知识蒸馏（KD）的NVIDIA Jetson Nano视网膜眼底图像异常检测
Cross-Architecture Knowledge Distillation (KD) for Retinal Fundus Image Anomaly Detection on NVIDIA Jetson Nano
AI总结：
本研究提出一种跨架构知识蒸馏框架，将基于I-JEPA预训练的ViT教师模型压缩为CNN学生模型，部署于NVIDIA Jetson Nano，在参数减少97.4%的情况下保留约93%诊断性能，实现89%分类准确率，为资源匮乏地区提供可扩展的视网膜疾病分诊方案。
摘要：
早期准确识别视网膜疾病对于防止视力衰退至关重要；然而，在资源匮乏的环境中，往往无法获得可靠的诊断设备。本项目提出通过开发一种使用跨架构知识蒸馏的轻量级、可边缘部署的疾病分类器来解决这一问题。我们首先训练一个高容量的视觉Transformer（ViT）教师模型，该模型使用I-JEPA自监督学习进行预训练，将眼底图像分为四类：正常、糖尿病视网膜病变、青光眼和白内障。在压缩为基于CNN的学生模型以部署在资源有限的条件（如NVIDIA Jetson Nano）时，我们保持了物联网（IoT）的关注重点。这是通过一个包含分区交叉注意力（PCA）投影器、分组线性（GL）投影器和多视图鲁棒训练方法的新颖框架实现的。教师模型的参数比学生模型多97.4%，学生模型实现了89%的分类准确率，并保留了教师模型约93%的诊断性能。临床分类行为的保留支持了我们方法的初始目标：在压缩ViT的同时保持准确性。我们的工作为资源不足地区的视网膜疾病提供了一个可扩展的、AI驱动的分诊解决方案的范例。
英文摘要：
Early and accurate identification of retinal ailments is crucial for averting ocular decline; however, access to dependable diagnostic devices is not often available in low-resourced settings. This project proposes to solve that by developing a lightweight, edge-device deployable disease classifier using cross-architecture knowledge distilling. We first train a high-capacity vision transformer (ViT) teacher model, pre-trained using I-JEPA self-supervised learning, to classify fundus images into four classes: Normal, Diabetic Retinopathy, Glaucoma, and Cataract. We kept an Internet of Things (IoT) focus when compressing to a CNN-based student model for deployment in resource-limited conditions, such as the NVIDIA Jetson Nano. This was accomplished using a novel framework which included a Partitioned Cross-Attention (PCA) projector, a Group-Wise Linear (GL) projector, and a multi-view robust training method. The teacher model has 97.4 percent more parameters than the student model, with it achieving 89 percent classification with a roughly 93 percent retention of the teacher model's diagnostic performance. The retention of clinical classification behavior supports our method's initial aim: compression of the ViT while retaining accuracy. Our work serves as an example of a scalable, AI-driven triage solution for retinal disorders in under-resourced areas.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
