音频视觉持续测试时间适应无需遗忘
Audio-Visual Continual Test-Time Adaptation without Forgetting
AI总结：
本文提出AVReCAP方法，在测试时无需源数据即可提升模型性能，通过动态检索最佳融合层参数减少灾难性遗忘。
摘要：
音频视觉持续测试时间适应涉及在测试时持续适应源音频视觉模型以适应非平稳目标域，其中任一模态可能分布偏移，阻碍在线跨模态学习并导致性能下降。尽管先前工作已解决此问题，但现有方法仍存在灾难性遗忘，即模型性能下降至甚至低于源模型。本文首先表明仅适应目标域的模态融合层不仅提升该域性能，还能增强后续域性能。基于融合层参数的强跨任务转移性，我们提出AVReCAP方法，在无需源数据的情况下提升模型测试性能。该方法通过选择性参数检索机制动态从缓冲区检索最佳融合层参数，仅使用少量测试数据即可完成参数检索、整合到模型中并适应当前测试分布，然后保存供未来使用。在包含单模态和双模态损坏的基准数据集上进行的广泛实验表明，所提AVReCAP方法在提升测试性能的同时显著减少灾难性遗忘。
英文摘要：
Audio-visual continual test-time adaptation involves continually adapting a source audio-visual model at test-time, to unlabeled non-stationary domains, where either or both modalities can be distributionally shifted, which hampers online cross-modal learning and eventually leads to poor accuracy. While previous works have tackled this problem, we find that SOTA methods suffer from catastrophic forgetting where the model's performance drops well below even the source model due to continual parameter updates at test-time. In this work, we first show that adapting only the modality fusion layer to a target domain not only improves performance on that domain but can also enhance performance on subsequent domains. Based on this strong cross-task transferability of the fusion layer's parameters, we propose a method, $\texttt{AVReCAP}$, that improves test-time performance of the models without access to any source data. Our approach works by using a selective parameter retrieval mechanism that dynamically retrieves the best fusion layer parameters from a buffer using only a small batch of test data. These parameters are then integrated into the model, adapted to the current test distribution, and saved back for future use. Extensive experiments on benchmark datasets involving unimodal and bimodal corruptions show our proposed $\texttt{AVReCAP}$ significantly outperforms existing methods while minimizing catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
