arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.00029cs.CVcs.IR
收藏

基于跨模态匹配的自动创意选择

Automatic Creative Selection with Cross-Modal Matching

University of Southern California(南加州大学);Apple(苹果公司)

Alex Kim, Jia Huang, Rob Monarch, Jerry Kwac, Anikesh Kamath, Parmeshwar Khurd, Kailash Thiyagarajan, Goodman Gu

AI总结:

本文提出基于微调LXMERT模型的应用图片与搜索词匹配方法,在广告主关联和人工标注两组真值上分别取得0.96和0.95的AUC,显著优于CLIP和Transformer+ResNet基线。

摘要:

应用开发者通过创建带有应用图片的产品页面并竞价搜索词来推广其应用。因此,应用图片与搜索词高度相关至关重要。解决这一问题需要一个图文匹配模型来预测所选图片与搜索词之间的匹配质量。在本工作中,我们提出了一种新颖的方法,基于微调预训练的LXMERT模型来匹配应用图片与搜索词。我们表明,与CLIP模型以及使用Transformer模型处理搜索词、ResNet模型处理图片的基线方法相比,我们显著提高了匹配准确率。我们使用两组标签评估了我们的方法:给定应用的广告主关联(图片,搜索词)对,以及(图片,搜索词)对相关性的人工评分。我们的方法在广告主关联真值上取得了0.96的AUC分数,分别比Transformer+ResNet基线和微调后的CLIP模型高出8%和14%。在人工标注真值上,我们的方法取得了0.95的AUC分数,分别比Transformer+ResNet基线和微调后的CLIP模型高出16%和17%。

英文摘要:

Application developers advertise their Apps by creating product pages with App images, and bidding on search terms. It is then crucial for App images to be highly relevant with the search terms. Solutions to this problem require an image-text matching model to predict the quality of the match between the chosen image and the search terms. In this work, we present a novel approach to matching an App image to search terms based on fine-tuning a pre-trained LXMERT model. We show that compared to the CLIP model and a baseline using a Transformer model for search terms, and a ResNet model for images, we significantly improve the matching accuracy. We evaluate our approach using two sets of labels: advertiser associated (image, search term) pairs for a given application, and human ratings for the relevance between (image, search term) pairs. Our approach achieves 0.96 AUC score for advertiser associated ground truth, outperforming the transformer+ResNet baseline and the fine-tuned CLIP model by 8% and 14%. For human labeled ground truth, our approach achieves 0.95 AUC score, outperforming the transformer+ResNet baseline and the fine-tuned CLIP model by 16% and 17%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑