扩展搜索相关性：用LLM生成的判断增强应用商店排名
Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments
AI总结：
针对应用商店排名中专家文本相关性标签稀缺的问题，通过微调LLM生成数百万标签，结合行为相关性优化排序器，显著提升Pareto前沿和转化率。
摘要：
大规模商业搜索系统优化相关性以驱动成功的会话，帮助用户找到他们想要的内容。为了最大化相关性，我们利用两个互补的目标：行为相关性（用户倾向于点击或下载的结果）和文本相关性（结果与查询的语义匹配）。一个持续的挑战是，相对于丰富的行为相关性标签，专家提供的文本相关性标签稀缺。我们首先通过系统评估LLM配置来解决这个问题，发现一个专门的、微调的模型在提供高度相关的标签方面显著优于一个更大的预训练模型。使用这个最优模型作为力量倍增器，我们生成了数百万个文本相关性标签以克服数据稀缺性。我们展示了用这些文本相关性标签增强我们的生产排序器会导致Pareto前沿显著外移：离线NDCG在行为相关性上改善，同时在文本相关性上也提高。这些离线收益通过在全球应用商店排序器上的A/B测试得到验证，该测试显示转化率统计上显著提高了+0.24%，其中最大的性能提升出现在尾部查询中，新的文本相关性标签在缺乏可靠行为相关性标签时提供了稳健的信号。
英文摘要：
Large-scale commercial search systems optimize for relevance to drive successful sessions that help users find what they are looking for. To maximize relevance, we leverage two complementary objectives: behavioral relevance (results users tend to click or download) and textual relevance (a result's semantic fit to the query). A persistent challenge is the scarcity of expert-provided textual relevance labels relative to abundant behavioral relevance labels. We first address this by systematically evaluating LLM configurations, finding that a specialized, fine-tuned model significantly outperforms a much larger pre-trained one in providing highly relevant labels. Using this optimal model as a force multiplier, we generate millions of textual relevance labels to overcome the data scarcity. We show that augmenting our production ranker with these textual relevance labels leads to a significant outward shift of the Pareto frontier: offline NDCG improves for behavioral relevance while simultaneously increasing for textual relevance. These offline gains were validated by a worldwide A/B test on the App Store ranker, which demonstrated a statistically significant +0.24% increase in conversion rate, with the most substantial performance gains occurring in tail queries, where the new textual relevance labels provide a robust signal in the absence of reliable behavioral relevance labels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
