SFT-KD-Recon：在磁共振图像重建中学习一个对学生友好的教师进行知识蒸馏
SFT-KD-Recon: Learning a Student-friendly Teacher for Knowledge Distillation in Magnetic Resonance Image Reconstruction
AI总结：
提出SFT-KD-Recon方法，通过联合训练教师与学生并引入模仿损失，使教师感知学生结构，从而提升磁共振图像重建中知识蒸馏的性能，显著缩小师生差距。
摘要：
用于磁共振成像（MRI）加速的深度级联架构在提供高质量重建方面已显示出显著的成功。然而，随着级联数量的增加，重建的改进趋于边际化，表明可能存在多余的模型容量。知识蒸馏（KD）是一种新兴的模型压缩技术，其中使用训练好的深度教师网络将知识蒸馏给较小的学生网络，使学生学会模仿教师的行为。大多数KD方法专注于使用预训练的、不了解学生模型的教师来有效训练学生。我们提出了SFT-KD-Recon，一种学生友好的教师训练方法，作为KD的前置步骤，与学生一起训练，使教师了解学生的结构和容量，并能够将教师的表示与学生对齐。在SFT中，教师与学生块的非展开分支配置联合训练，使用三个损失项——教师重建损失、学生重建损失和师生模仿损失，随后进行学生的KD。我们在脑部和心脏数据集上，以4倍和5倍欠采样进行MRI加速的广泛实验，使用所提出的方法作为前置步骤，对五种KD方法进行了评估。我们考虑DC-CNN架构，设置教师为D5C5（141765个参数），学生为D3C5（49285个参数），表示压缩比为2.87:1。结果表明：（i）我们的方法一致地改进了KD方法，提高了重建性能和图像质量；（ii）使用我们的方法蒸馏的学生与教师具有竞争力，性能差距从0.53 dB减小到0.03 dB。
英文摘要：
Deep cascaded architectures for magnetic resonance imaging (MRI) acceleration have shown remarkable success in providing high-quality reconstruction. However, as the number of cascades increases, the improvements in reconstruction tend to become marginal, indicating possible excess model capacity. Knowledge distillation (KD) is an emerging technique to compress these models, in which a trained deep teacher network is used to distill knowledge to a smaller student network such that the student learns to mimic the behavior of the teacher. Most KD methods focus on effectively training the student with a pre-trained teacher unaware of the student model. We propose SFT-KD-Recon, a student-friendly teacher training approach along with the student as a prior step to KD to make the teacher aware of the structure and capacity of the student and enable aligning the representations of the teacher with the student. In SFT, the teacher is jointly trained with the unfolded branch configurations of the student blocks using three loss terms - teacher-reconstruction loss, student-reconstruction loss, and teacher-student imitation loss, followed by KD of the student. We perform extensive experiments for MRI acceleration in 4x and 5x under-sampling on the brain and cardiac datasets on five KD methods using the proposed approach as a prior step. We consider the DC-CNN architecture and setup teacher as D5C5 (141765 parameters), and student as D3C5 (49285 parameters), denoting a compression of 2.87:1. Results show that (i) our approach consistently improves the KD methods with improved reconstruction performance and image quality, and (ii) the student distilled using our approach is competitive with the teacher, with the performance gap reduced from 0.53 dB to 0.03 dB.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
