基于双模块记忆化的持续指代表达理解
Continual Referring Expression Comprehension via Dual Modular Memorization
AI总结：
针对现有REC模型需预先给定训练数据的问题，提出持续指代表达理解（CREC）新设定，并设计了双模块记忆化（DMM）方法，通过隐式与显式记忆缓解灾难性遗忘，在构建的三个基准上取得显著效果。
摘要：
指代表达理解（Referring Expression Comprehension, REC）旨在定位由自然语言表达式描述的给定对象的图像区域。尽管已展现出良好的性能，但现有的REC算法强烈假设输入模型的训练数据是预先给定的，这降低了其在真实世界场景中的实用性。在本文中，我们提出了持续指代表达理解（Continual Referring Expression Comprehension, CREC），一种新的REC设定，其中模型在持续到来的任务流上进行学习。为了在序列任务上持续提升模型而不遗忘已学知识且无需从头反复训练，我们提出了一种名为双模块记忆化（Dual Modular Memorization, DMM）的有效基线方法，该方法通过两个记忆化模块来缓解灾难性遗忘问题：隐式记忆和显式记忆。具体而言，前一个模块旨在学习新任务时约束旧任务中已学习的重要参数发生剧烈变化；而后一个模块维护一个缓冲池，以动态选择并存储每个已见任务的代表性样本用于未来重演。我们通过分别将三个广泛使用的REC数据集RefCOCO、RefCOCO+和RefCOCOg重新划分为序列任务，为新的CREC设定创建了三个基准。在所构建的基准上进行的大量实验表明，基于两种流行的REC骨干网络，我们的DMM方法显著优于其他替代方案。我们公开了源代码和基准以促进该领域的未来进展：https://github.com/zackschen/DMM。
英文摘要：
Referring Expression Comprehension (REC) aims to localize an image region of a given object described by a natural-language expression. While promising performance has been demonstrated, existing REC algorithms make a strong assumption that training data feeding into a model are given upfront, which degrades its practicality for real-world scenarios. In this paper, we propose Continual Referring Expression Comprehension (CREC), a new setting for REC, where a model is learning on a stream of incoming tasks. In order to continuously improve the model on sequential tasks without forgetting prior learned knowledge and without repeatedly re-training from a scratch, we propose an effective baseline method named Dual Modular Memorization (DMM), which alleviates the problem of catastrophic forgetting by two memorization modules: Implicit-Memory and Explicit-Memory. Specifically, the former module aims to constrain drastic changes to important parameters learned on old tasks when learning a new task; while the latter module maintains a buffer pool to dynamically select and store representative samples of each seen task for future rehearsal. We create three benchmarks for the new CREC setting, by respectively re-splitting three widely-used REC datasets RefCOCO, RefCOCO+ and RefCOCOg into sequential tasks. Extensive experiments on the constructed benchmarks demonstrate that our DMM method significantly outperforms other alternatives, based on two popular REC backbones. We make the source code and benchmarks publicly available to foster future progress in this field: https://github.com/zackschen/DMM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
