目录短语定位（CPG）：面向电商视觉语言应用的产品图像中产品文本属性定位
Catalog Phrase Grounding (CPG): Grounding of Product Textual Attributes in Product Images for e-commerce Vision-Language Applications
AI总结：
提出CPG模型，利用自监督多模态Transformer将产品文本属性定位到图像区域，通过教师模型知识迁移，在品牌匹配任务中提升召回率平均5%。
摘要：
我们提出了目录短语定位（CPG），一种能够将产品文本数据（标题、品牌）关联到产品图像中相应区域（孤立产品区域、品牌标志区域）的模型，用于电商视觉语言应用。我们采用了一种最先进的调制多模态Transformer编码器-解码器架构，统一了目标检测和短语定位。我们以自监督方式训练该模型，使用了从电商网站合成的230万图像-文本对。自监督数据带有高置信度伪标签，这些伪标签由教师模型组合生成：一个预训练的通用领域短语定位模型（如MDETR）和一个专门的标志检测模型。这使得CPG作为学生模型，能够从这些基础模型中受益，结合通用领域知识和专门知识进行知识迁移。除了直接的目录短语定位任务外，我们还可以将CPG表示作为机器学习特征，纳入需要深度产品语义理解的下游目录应用中。我们在产品-品牌匹配（一项具有挑战性的电商应用）上的实验表明，将CPG表示纳入现有生产集成系统，在固定95%精确率下，全球所有国家的召回率平均提升5%（单个国家最大提升达11%），优于其他替代方案，包括标志检测教师模型和ResNet50。
英文摘要：
We present Catalog Phrase Grounding (CPG), a model that can associate product textual data (title, brands) into corresponding regions of product images (isolated product region, brand logo region) for e-commerce vision-language applications. We use a state-of-the-art modulated multimodal transformer encoder-decoder architecture unifying object detection and phrase-grounding. We train the model in self-supervised fashion with 2.3 million image-text pairs synthesized from an e-commerce site. The self-supervision data is annotated with high-confidence pseudo-labels generated with a combination of teacher models: a pre-trained general domain phrase grounding model (e.g. MDETR) and a specialized logo detection model. This allows CPG, as a student model, to benefit from transfer knowledge from these base models combining general-domain knowledge and specialized knowledge. Beyond immediate catalog phrase grounding tasks, we can benefit from CPG representations by incorporating them as ML features into downstream catalog applications that require deep semantic understanding of products. Our experiments on product-brand matching, a challenging e-commerce application, show that incorporating CPG representations into the existing production ensemble system leads to on average 5% recall improvement across all countries globally (with the largest lift of 11% in a single country) at fixed 95% precision, outperforming other alternatives including a logo detection teacher model and ResNet50.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
