Kaleido-BERT：时尚领域的视觉语言预训练
Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
摘要：
我们提出了一个新的视觉语言（VL）预训练模型称为Kaleido-BERT，它引入了一种新的kaleido策略，用于Transformers的时尚跨模态表示。相对于最近VL模型的随机掩蔽策略，我们设计了对齐引导掩蔽，共同关注更多的图像-文本语义关系。为此，我们开展了五项新任务，即：旋转，拼图，伪装，灰色到彩色，和空白到彩色的自我监督VL预训练在不同规模的补丁。Kaleido-BERT概念简单，易于扩展到现有的BERT框架，在文本检索（R@1：4.03%绝对改进）、图像检索（R@1：7.13%绝对改进）、类别识别（ACC：3.28%绝对值），和时尚字幕（蓝色4：1.2绝对imv。）。我们验证了Kaleido-BERT在各种电子商务网站上的效率，展示了其在现实世界应用中的更广泛潜力。
英文摘要：
We present a new vision-language (VL) pre-training model dubbed Kaleido-BERT, which introduces a novel kaleido strategy for fashion cross-modality representations from transformers. In contrast to random masking strategy of recent VL models, we design alignment guided masking to jointly focus more on image-text semantic relations. To this end, we carry out five novel tasks, i.e., rotation, jigsaw, camouflage, grey-to-color, and blank-to-color for self-supervised VL pre-training at patches of different scale. Kaleido-BERT is conceptually simple and easy to extend to the existing BERT framework, it attains new state-of-the-art results by large margins on four downstream tasks, including text retrieval (R@1: 4.03% absolute improvement), image retrieval (R@1: 7.13% abs imv.), category recognition (ACC: 3.28% abs imv.), and fashion captioning (Bleu4: 1.2 abs imv.). We validate the efficiency of Kaleido-BERT on a wide range of e-commerical websites, demonstrating its broader potential in real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
