arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.07913cs.CV
收藏

大规模预训练时代的幸存者:一阶段指代表达式理解的经验研究

A Survivor in the Era of Large-Scale Pretraining: An Empirical Study of One-Stage Referring Expression Comprehension

Xiamen University(厦门大学);Peng Cheng Laboratory(鹏城实验室)

Gen Luo, Yiyi Zhou, Jiamu Sun, Xiaoshuai Sun, Rongrong Ji

AI总结:

本文对一阶段指代表达式理解(REC)进行经验研究,构建简单网络SimREC并消融42种设计,发现多尺度特征、数据增强等关键因素及反常规认知,以更少参数超越大规模预训练模型。

摘要:

现有的一阶段指代表达式理解(REC)工作大多聚焦于多模态融合与推理,而其他因素对该任务的影响缺乏深入探索。为填补这一空白,本文进行了经验研究。具体而言,我们首先构建了一个非常简单的REC网络SimREC,并消融了42种候选设计/设置,涵盖从网络设计到模型训练的一阶段REC全过程。随后,我们在三个REC基准数据集上进行了超过100次实验。大量实验结果不仅展示了除多模态融合外影响REC性能的关键因素(如多尺度特征和数据增强),还得出了一些与常规认知相悖的发现。例如,作为视觉与语言(V&L)任务,REC确实较少受语言先验影响。此外,通过合理结合这些发现,我们大幅提升了SimREC的性能,例如在RefCOCO+上提升+27.12%,超越了所有现有REC方法。但最令人鼓舞的发现是,尽管训练开销和参数量大幅减少,SimREC仍能比一系列大规模预训练模型(如UNITER和VILLA)取得更好性能,描绘了REC在现有V&L研究中的特殊角色。

英文摘要:

Most of the existing work in one-stage referring expression comprehension (REC) mainly focuses on multi-modal fusion and reasoning, while the influence of other factors in this task lacks in-depth exploration. To fill this gap, we conduct an empirical study in this paper. Concretely, we first build a very simple REC network called SimREC, and ablate 42 candidate designs/settings, which covers the entire process of one-stage REC from network design to model training. Afterwards, we conduct over 100 experimental trials on three benchmark datasets of REC. The extensive experimental results not only show the key factors that affect REC performance in addition to multi-modal fusion, e.g., multi-scale features and data augmentation, but also yield some findings that run counter to conventional understanding. For example, as a vision and language (V&L) task, REC does is less impacted by language prior. In addition, with a proper combination of these findings, we can improve the performance of SimREC by a large margin, e.g., +27.12% on RefCOCO+, which outperforms all existing REC methods. But the most encouraging finding is that with much less training overhead and parameters, SimREC can still achieve better performance than a set of large-scale pre-trained models, e.g., UNITER and VILLA, portraying the special role of REC in existing V&L research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑