FlashEvaluator: 通过并行评估扩展搜索空间
FlashEvaluator: Expanding Search Space with Parallel Sequence-Level Evaluation
AI总结：
FlashEvaluator通过并行评估实现跨序列信息共享，提升推荐和NLP任务的效率与准确性。
摘要：
生成器-评估器（G-E）框架，即从生成器中评估K个序列并根据评估器评分选择排名第一的序列，是推荐系统（RecSys）和自然语言处理（NLP）等任务中的基础范式。传统评估器独立处理序列，存在两个主要局限：（1）缺乏显式的跨序列比较，导致准确性不足；（2）与线性复杂度O(K)的平行化能力差，导致资源利用率低，影响吞吐量和延迟。为解决这些挑战，我们提出了FlashEvaluator，它实现了跨序列标记信息共享，并在单次前向传递中处理所有序列。这带来了亚线性计算复杂度，提高了系统的效率，并支持直接的跨序列比较，从而提高了选择准确性。本文还提供了理论证明和在推荐和NLP任务中的广泛实验，展示了与传统方法相比的明显优势。值得注意的是，FlashEvaluator已在快手的在线推荐系统中部署，在实践中带来了显著且持续的收益增长。
英文摘要：
The Generator-Evaluator (G-E) framework generates K candidate sequences and uses an evaluator to select the highest-scoring one, which is widely used in recommender systems (RecSys) and natural language processing (NLP). Existing evaluators commonly score candidates independently. Although such evaluations can be batched, independent scoring neither models interactions among candidates nor eliminates repeated computation of request-level context and recurring candidate elements, causing the total evaluation work to grow approximately linearly with K. To handle with, we propose FlashEvaluator, a joint evaluator that scores all candidate sequences in a single forward pass. FlashEvaluator factorizes evaluation into shared request-level encoding, reusable candidate-side computation, sequence assembly by indexing, and cross-sequence interaction for setwise comparison. We call this request-local reuse scheme QKV-Cache: inspired by autoregressive KV caching, it reuses context-side key/value representations across candidate sequences and, when candidate elements recur, reuses their request-conditioned representations on the query side. In repeated-item settings, the dominant item-encoding cost therefore depends on the number of distinct items rather than their total occurrences across sequences, reducing the marginal cost of evaluating additional candidates. We provide a computational analysis and evaluate FlashEvaluator on recommendation and text summarization. The results show lower latency and higher throughput with competitive recommendation and summarization quality. In an online deployment at Kuaishou with K=50, FlashEvaluator reduces inference latency by 44% and increases QPS by 114% relative to the production baseline, while yielding statistically significant gains in retention, engagement, and ecosystem metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
