FAAR：通过自动排名选择实现高效的频率感知多任务微调
FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection
AI总结：
本文提出FAAR方法，通过性能驱动的排名缩减和任务频谱金字塔解码器，在多任务学习中提升准确性和效率，相比传统方法减少参数达9倍。
摘要：
适配预训练在大规模数据集上的模型是快速达到下游任务强大性能的已知方法。然而，随着最先进模型的增长，传统全微调变得不合适且困难，尤其是在多任务学习（MTL）中，成本随任务数量增加而增加。因此，最近的研究利用低秩适应进行参数高效微调（PEFT）以显著减少可训练参数的数量。然而，这些现有方法使用单一固定秩，可能对不同任务或MTL架构中的位置不最优。此外，这些方法未能学习捕捉跨任务关系的空间信息，有助于改进多样任务预测。本文介绍频率感知和自动排名（FAAR）用于高效的MTL微调。我们的方法引入了性能驱动的排名缩减（PDRS）以在每个适配器位置和每个任务上分配最优的秩。此外，通过分析图像频率谱，FAAR提出任务频谱金字塔解码器（TS-PD），将输入特定的上下文注入空间偏差学习以更好地反映跨任务关系。在密集视觉任务基准上的实验显示，与其它PEFT方法相比，我们的方法在准确性和效率方面均占优。FAAR将参数数量减少至传统MTL微调的9倍，同时提升整体性能。我们的代码已提供。
英文摘要：
Adapting models pre-trained on large-scale datasets is a proven way to reach strong performance quickly for down-stream tasks. However, the growth of state-of-the-art mod-els makes traditional full fine-tuning unsuitable and difficult, especially for multi-task learning (MTL) where cost scales with the number of tasks. As a result, recent studies investigate parameter-efficient fine-tuning (PEFT) using low-rank adaptation to significantly reduce the number of trainable parameters. However, these existing methods use a single, fixed rank, which may not be optimal for differ-ent tasks or positions in the MTL architecture. Moreover, these methods fail to learn spatial information that cap-tures inter-task relationships and helps to improve diverse task predictions. This paper introduces Frequency-Aware and Automatic Rank (FAAR) for efficient MTL fine-tuning. Our method introduces Performance-Driven Rank Shrink-ing (PDRS) to allocate the optimal rank per adapter location and per task. Moreover, by analyzing the image frequency spectrum, FAAR proposes a Task-Spectral Pyramidal Decoder (TS-PD) that injects input-specific context into spatial bias learning to better reflect cross-task relationships. Experiments performed on dense visual task benchmarks show the superiority of our method in terms of both accuracy and efficiency compared to other PEFT methods in MTL. FAAR reduces the number of parameters by up to 9 times compared to traditional MTL fine-tuning whilst improving overall performance. Our code is available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
