面向少样本器官分割的正则化低秩适配
Regularized Low-Rank Adaptation for Few-Shot Organ Segmentation
AI总结：
本文提出一种基于l_1稀疏正则化和近端优化的动态低秩适配方法，用于少样本器官分割，能自动学习任务适配秩，在基础器官和新器官任务上显著优于标准LoRA及其他PEFT方法。
摘要：
预训练基础模型的参数高效微调（PEFT）因其有效性和计算效率，在医学影像领域日益受到关注。在这些方法中，低秩适配（Low-Rank Adaptation, LoRA）是一种基于适配本质上发生在低维子空间中这一假设的显著方法。尽管LoRA已展现出良好的性能，但其实现需要一个固定且不可更改的秩，而考虑到每个医学影像下游任务的独特复杂性和需求，选择该秩可能颇具挑战。受自然图像处理领域进展的启发，我们提出一种用于医学图像分割的新方法，该方法在适配过程中动态调整内在秩。将可训练权重矩阵的低秩表示视为奇异值分解，我们在损失函数中引入l_1稀疏正则化项，并使用近端优化器进行求解。该正则化项可视为对分解秩的惩罚。因此，对其最小化能够自动找到适应任务的秩。我们在现实的少样本微调设置下评估了我们的方法，首先将其与标准LoRA进行比较，然后在两个可区分任务（基础器官和新器官）上与若干其他PEFT方法进行比较。大量实验表明，我们的方法带来了显著的性能提升，突显了其在面对次优秩初始化时的高效性和鲁棒性。我们的代码已公开：https://github.com/ghassenbaklouti/ARENA
英文摘要：
Parameter-efficient fine-tuning (PEFT) of pre-trained foundation models is increasingly attracting interest in medical imaging due to its effectiveness and computational efficiency. Among these methods, Low-Rank Adaptation (LoRA) is a notable approach based on the assumption that the adaptation inherently occurs in a low-dimensional subspace. While it has shown good performance, its implementation requires a fixed and unalterable rank, which might be challenging to select given the unique complexities and requirements of each medical imaging downstream task. Inspired by advancements in natural image processing, we introduce a novel approach for medical image segmentation that dynamically adjusts the intrinsic rank during adaptation. Viewing the low-rank representation of the trainable weight matrices as a singular value decomposition, we introduce an l_1 sparsity regularizer to the loss function, and tackle it with a proximal optimizer. The regularizer could be viewed as a penalty on the decomposition rank. Hence, its minimization enables to find task-adapted ranks automatically. Our method is evaluated in a realistic few-shot fine-tuning setting, where we compare it first to the standard LoRA and then to several other PEFT methods across two distinguishable tasks: base organs and novel organs. Our extensive experiments demonstrate the significant performance improvements driven by our method, highlighting its efficiency and robustness against suboptimal rank initialization. Our code is publicly available: https://github.com/ghassenbaklouti/ARENA
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
