主观检索系统的产品化
Towards Productionizing Subjective Search Systems
摘要：
现有的电子商务搜索引擎通常只支持对客观属性（如价格和位置）的搜索，而对更理想的主观属性（如浪漫氛围和工作生活平衡）则无法搜索。我们发现，Recruit Group也是如此，该公司经营着广泛的在线预订和搜索服务，包括工作、旅游、住房、新娘、餐饮、美容，每项服务都是日本最大的，如果不是国际上的话。我们提出了我们的生产化最近的主观搜索原型（OpineDB）Megagon实验室为招聘集团开发的进展。OpineDB中的几个组件得到了增强，以满足生产需求，包括添加一个BERT语言模型，该模型是在大量酒店领域评论语料库上预先训练的。我们还发现，将系统生产化的挑战不仅仅是增强组件。特别是，在生产质量系统中的一个重要需求是测量搜索质量的适当方法，当搜索结果是主观的时，这是非常棘手的。这导致从头开始创建一个高质量的基准数据集，涉及用户访谈的600多个查询和120，000多个查询实体相关性标签的集合。此外，我们发现，现有的搜索算法不符合搜索质量标准所要求的生产系统。因此，我们通过微调几种搜索算法并将它们结合在学习排名框架下来增强排名模型。该模型实现了5%-10%的整体精度提高和90%以上的基准测试查询，使这些查询准备AB测试的精度。虽然一些增强可以立即应用于其他垂直领域，但我们的经验表明，基准测试和微调排名算法是针对每个领域的，无法避免。
英文摘要：
Existing e-commerce search engines typically support search only over objective attributes, such as price and locations, leaving the more desirable subjective attributes, such as romantic vibe and worklife balance unsearchable. We found that this is also the case for Recruit Group, which operates a wide range of online booking and search services, including jobs, travel, housing, bridal, dining, beauty, and where each service is among the biggest in Japan, if not internationally. We present our progress towards productionizing a recent subjective search prototype (OpineDB) developed by Megagon Labs for Recruit Group. Several components within OpineDB are enhanced to satisfy production demands, including adding a BERT language model pre-trained on massive hospitality domain review corpora. We also found that the challenges of productionizing the system are beyond enhancing the components. In particular, an important requirement in production-quality systems is to instrument a proper way of measuring the search quality, which is extremely tricky when the search results are subjective. This led to the creation of a high-quality benchmark dataset from scratch, involving over 600 queries by user interviews and a collection of more than 120,000 query-entity relevancy labels. Also, we found that the existing search algorithms do not meet the search quality standard required by production systems. Consequently, we enhanced the ranking model by fine-tuning several search algorithms and combining them under a learning-to-rank framework. The model achieves 5%-10% overall precision improvement and 90+% precision on more than half of the benchmark testing queries making these queries ready for AB-testing. While some enhancements can be immediately applied to other verticals, our experience reveals that benchmarking and fine-tuning ranking algorithms are specific to each domain and cannot be avoided.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
