arXiv 2312.124802024-03-27cs.CV

Continual-MAE：用于持续测试时自适应的自适应分布掩码自编码器

Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation

National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；School of Computer Science（计算机学院）；Peking University（北京大学）；CUHK（香港中文大学）；University of Science Technology of China（中国科学技术大学）

Jiaming Liu, Ran Xu, Senqiao Yang, Renrui Zhang, Qizhe Zhang, Zehui Chen, Yandong Guo, Shanghang Zhang

备注：: Accepted by CVPR2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对持续测试时自适应中预测失准和噪声伪标签问题，提出自适应分布掩码自编码器（ADMA），通过分布感知掩码机制与重建手工特征描述符增强目标域知识提取，在分类与分割任务上达到SOTA。

摘要：

持续测试时自适应（CTTA）旨在将源预训练模型迁移至不断变化的目标分布，以应对现实世界的动态性。现有CTTA方法主要依赖熵最小化或师生伪标签方案在无标签目标域中进行知识提取。然而，动态数据分布会导致现有自监督学习方法出现预测失准和噪声伪标签，阻碍了在持续自适应过程中有效缓解错误累积和灾难性遗忘问题。为解决这些问题，我们提出了一种持续自监督方法，即自适应分布掩码自编码器（ADMA），该方法增强了目标域知识提取，同时缓解了分布偏移的累积。具体而言，我们提出分布感知掩码（DaM）机制来自适应地采样掩码位置，随后在掩码目标样本与原始目标样本之间建立一致性约束。此外，对于掩码token，我们利用高效解码器重建手工特征描述符（例如方向梯度直方图），利用其不变性来增强任务相关表示。通过在四个广泛认可的基准上进行大量实验，我们提出的方法在分类和分割CTTA任务中均达到了最先进的性能。项目主页：https://sites.google.com/view/continual-mae/home。

英文摘要：

Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions, addressing real-world dynamism. Existing CTTA methods mainly rely on entropy minimization or teacher-student pseudo-labeling schemes for knowledge extraction in unlabeled target domains. However, dynamic data distributions cause miscalibrated predictions and noisy pseudo-labels in existing self-supervised learning methods, hindering the effective mitigation of error accumulation and catastrophic forgetting problems during the continual adaptation process. To tackle these issues, we propose a continual self-supervised method, Adaptive Distribution Masked Autoencoders (ADMA), which enhances the extraction of target domain knowledge while mitigating the accumulation of distribution shifts. Specifically, we propose a Distribution-aware Masking (DaM) mechanism to adaptively sample masked positions, followed by establishing consistency constraints between the masked target samples and the original target samples. Additionally, for masked tokens, we utilize an efficient decoder to reconstruct a hand-crafted feature descriptor (e.g., Histograms of Oriented Gradients), leveraging its invariant properties to boost task-relevant representations. Through conducting extensive experiments on four widely recognized benchmarks, our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks. Our project page: https://sites.google.com/view/continual-mae/home.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。