DAM：用于持续视频问答学习的动态适配器合并
DAM: Dynamic Adapter Merging for Continual Video QA Learning
AI总结：
提出参数高效的持续视频问答学习方法DAM，通过动态合并特定适配器权重并利用非参数路由器，有效缓解灾难性遗忘并实现跨域知识共享，在多个数据集上显著超越现有方法。
摘要：
我们提出了一种用于持续视频问答（VidQA）学习的参数高效方法。我们的方法名为DAM，使用所提出的动态适配器合并来（i）缓解灾难性遗忘，（ii）实现对持续到达数据集的高效适应，（iii）处理推理期间来自未知数据集的输入，以及（iv）实现跨相似数据集域的知识共享。给定一组持续流式传输的VidQA数据集，我们顺序地为每个数据集训练特定于数据集的适配器，同时冻结大型预训练视频语言主干模型的参数。在推理期间，给定来自未知域的视频-问题样本，我们的方法首先使用所提出的非参数路由器函数为每个适配器计算一个概率，反映该适配器与当前视频-问题输入实例的相关程度。随后，所提出的动态适配器合并方案将所有适配器权重聚合为一个为该特定测试样本量身定制的新适配器实例，以计算最终的VidQA预测，从而缓解不准确路由器预测的影响并促进跨域知识共享。我们的DAM模型在跨越各个域的6个VidQA数据集上超越了先前的最先进持续学习方法9.1%，同时表现出减少1.9%的遗忘。我们进一步将DAM扩展到持续图像分类和图像问答，并以巨大优势超越了先前的方法。代码公开于：https://github.com/klauscc/DAM
英文摘要：
We present a parameter-efficient method for continual video question-answering (VidQA) learning. Our method, named DAM, uses the proposed Dynamic Adapter Merging to (i) mitigate catastrophic forgetting, (ii) enable efficient adaptation to continually arriving datasets, (iii) handle inputs from unknown datasets during inference, and (iv) enable knowledge sharing across similar dataset domains. Given a set of continually streaming VidQA datasets, we sequentially train dataset-specific adapters for each dataset while freezing the parameters of a large pretrained video-language backbone. During inference, given a video-question sample from an unknown domain, our method first uses the proposed non-parametric router function to compute a probability for each adapter, reflecting how relevant that adapter is to the current video-question input instance. Subsequently, the proposed dynamic adapter merging scheme aggregates all the adapter weights into a new adapter instance tailored for that particular test sample to compute the final VidQA prediction, mitigating the impact of inaccurate router predictions and facilitating knowledge sharing across domains. Our DAM model outperforms prior state-of-the-art continual learning approaches by 9.1% while exhibiting 1.9% less forgetting on 6 VidQA datasets spanning various domains. We further extend DAM to continual image classification and image QA and outperform prior methods by a large margin. The code is publicly available at: https://github.com/klauscc/DAM
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
