频率引导掩码增强视觉自监督学习
Frequency-Guided Masking for Enhanced Vision Self-Supervised Learning
AI总结：
针对现有频率掩码自监督学习忽略频率响应变异性和微调适配困难的问题，提出FOLK方法，通过自适应频率选择和知识蒸馏双分支框架，在分类、少样本和分割任务上取得先进性能。
摘要：
我们提出了一种新颖的基于频率的自监督学习（SSL）方法，显著提升了其预训练效果。先前相关工作在输入图像中掩码掉预定义的频率，并采用重建损失来预训练模型。尽管取得了有前景的结果，但我们在论文中指出，这种实现存在两个根本性局限。第一，使用预定义频率忽略了图像频率响应的变异性。第二，使用频率滤波图像预训练的模型，在微调阶段需要相对更多的数据来适应自然图像。为解决这些缺陷，我们提出了傅里叶变换压缩与自知识蒸馏（FOLK）方法，整合了两个专门的设计。首先，受图像压缩启发，我们根据图像频率响应自适应地选择被掩码的频率，从而为预训练创建更合适的SSL任务。其次，我们采用由知识蒸馏驱动的双分支框架，使模型能够同时以滤波图像和原始图像作为输入，大幅减轻下游任务的负担。我们的实验结果证明了FOLK在多种下游任务（包括图像分类、少样本学习和语义分割）中取得与许多最先进SSL方法相当性能的有效性。
英文摘要：
We present a novel frequency-based Self-Supervised Learning (SSL) approach that significantly enhances its efficacy for pre-training. Prior work in this direction masks out pre-defined frequencies in the input image and employs a reconstruction loss to pre-train the model. While achieving promising results, such an implementation has two fundamental limitations as identified in our paper. First, using pre-defined frequencies overlooks the variability of image frequency responses. Second, pre-trained with frequency-filtered images, the resulting model needs relatively more data to adapt to naturally looking images during fine-tuning. To address these drawbacks, we propose FOurier transform compression with seLf-Knowledge distillation (FOLK), integrating two dedicated ideas. First, inspired by image compression, we adaptively select the masked-out frequencies based on image frequency responses, creating more suitable SSL tasks for pre-training. Second, we employ a two-branch framework empowered by knowledge distillation, enabling the model to take both the filtered and original images as input, largely reducing the burden of downstream tasks. Our experimental results demonstrate the effectiveness of FOLK in achieving competitive performance to many state-of-the-art SSL methods across various downstream tasks, including image classification, few-shot learning, and semantic segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
