评判还是不评判：在eBay使用LLM评判广告主关键词相关性
To Judge or not to Judge: Using LLM Judgements for Advertiser Keyphrase Relevance at eBay
AI总结：
本研究针对eBay广告主关键词相关性过滤问题，提出使用LLM作为评判者作为卖家判断的可扩展代理，以在卖家、广告和搜索三个动态系统间实现更好和谐，并通过案例研究验证其有效性。
摘要：
电子商务卖家基于其库存被推荐关键词，并针对这些关键词投放广告以提升买家参与度（点击/销售）。广告主关键词的相关性在防止搜索系统被大量不相关商品淹没方面发挥着重要作用，这些不相关商品在拍卖中争夺注意力，同时也有助于维护卖家的良好感知。在本工作中，我们描述了在点击/销售/搜索相关性信号上训练广告主关键词相关性过滤模型的缺陷，以及与人判断对齐的重要性，因为卖家有权采纳或拒绝上述关键词推荐。在本研究中，我们将广告主关键词相关性构建为三个动态系统之间的复杂交互——卖家判断，它影响卖家对我们产品的采纳；广告，它提供可供竞价的关键词；以及搜索，它为相同关键词举办拍卖。本研究通过eBay广告部门的案例研究讨论了使用人判断的实践性，并证明大规模使用LLM作为评判者作为卖家判断的可扩展代理来训练我们的相关性模型，能够在三个系统之间实现更好的和谐——前提是它们受到一个基于业务指标、细致入微的评估框架的约束。
英文摘要：
E-commerce sellers are recommended keyphrases based on their inventory on which they advertise to increase buyer engagement (clicks/sales). The relevance of advertiser keyphrases plays an important role in preventing the inundation of search systems with numerous irrelevant items that compete for attention in auctions, in addition to maintaining a healthy seller perception. In this work, we describe the shortcomings of training Advertiser keyphrase relevance filter models on click/sales/search relevance signals and the importance of aligning with human judgment, as sellers have the power to adopt or reject said keyphrase recommendations. In this study, we frame Advertiser keyphrase relevance as a complex interaction between 3 dynamical systems -- seller judgment, which influences seller adoption of our product, Advertising, which provides the keyphrases to bid on, and Search, who holds the auctions for the same keyphrases. This study discusses the practicalities of using human judgment via a case study at eBay Advertising and demonstrate that using LLM-as-a-judge en-masse as a scalable proxy for seller judgment to train our relevance models achieves a better harmony across the three systems -- provided that they are bound by a meticulous evaluation framework grounded in business metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
