arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.04842cs.CVcs.AI
收藏

PIVOT:用于视频持续学习的提示方法

PIVOT: Prompting for Video Continual Learning

Pontificia Universidad Católica de Chile(智利天主教大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);University of Pisa(比萨大学);Adobe Research(Adobe研究院)

Andrés Villa, Juan León Alcázar, Motasem Alfarra, Kumail Alhamoud, Julio Hurtado, Fabian Caba Heilbron, Alvaro Soto, Bernard Ghanem

AI总结:

针对视频数据持续学习问题,提出PIVOT方法,利用图像域预训练模型知识并采用提示机制,无需领域内预训练即可减少参数与遗忘,在20任务ActivityNet上将最先进方法提升27%。

摘要:

现代机器学习流程因数据可用性、存储配额、隐私法规和昂贵的标注过程而受到限制。这些约束使得在动态标注集上训练和更新大规模模型变得困难或不可能。持续学习直接解决这一问题,其最终目标是设计出使深度神经网络能有效学习新(未见)类别相关模式,同时不显著改变其在已学类别上性能的方法。本文针对视频数据的持续学习问题,提出了PIVOT方法,该方法利用图像域预训练模型中的丰富知识,从而减少可训练参数数量及相关的遗忘。与以往方法不同,这是首个有效利用提示机制且无需任何领域内预训练的持续学习方法。实验表明,在20任务的ActivityNet设置上,PIVOT将最先进方法显著提升了27%。

英文摘要:

Modern machine learning pipelines are limited due to data availability, storage quotas, privacy regulations, and expensive annotation processes. These constraints make it difficult or impossible to train and update large-scale models on such dynamic annotated sets. Continual learning directly approaches this problem, with the ultimate goal of devising methods where a deep neural network effectively learns relevant patterns for new (unseen) classes, without significantly altering its performance on previously learned ones. In this paper, we address the problem of continual learning for video data. We introduce PIVOT, a novel method that leverages extensive knowledge in pre-trained models from the image domain, thereby reducing the number of trainable parameters and the associated forgetting. Unlike previous methods, ours is the first approach that effectively uses prompting mechanisms for continual learning without any in-domain pre-training. Our experiments show that PIVOT improves state-of-the-art methods by a significant 27% on the 20-task ActivityNet setup.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑