QueryFormer：KDD Cup 2026腾讯UniRec挑战赛的获胜方案
QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge
查看机构详情
- Wuhan University（武汉大学）
- Sun Yat-sen University（中山大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
QueryFormer通过可堆叠的统一字段-序列块和交叉注意力生成查询，解决pCVR预测中的特征交互与序列行为建模，在KDD Cup 2026腾讯UniRec挑战赛工业赛道夺冠，AUC达0.83254。
中文摘要 AI 辅助
点击后转化率（pCVR）预测需要联合建模特征交互和序列化用户行为。KDD Cup 2026腾讯UniRec挑战赛要求一个统一的架构同时处理这两方面。我们观察到，现有的统一架构通常使用基于投影的多层感知机（MLPs）生成查询令牌——即中央信息枢纽——而没有显式的令牌到查询的注意力机制来细化查询侧。我们提出QueryFormer，其核心是一个可堆叠的统一字段-序列块，该块桥接了非序列的多字段特征和行为序列，并提供了关于视图宽度$H$、模型宽度、深度、数据和计算量的延迟感知扩展研究。该块通过交叉注意力生成查询，并将序列查询打包到共享参数注意力中。QueryFormer在工业赛道中获得第一名，官方测试集ROC曲线下面积（AUC）达到0.83254；赛后适度扩展达到0.832713。在我们的网格中，$H$扩展将验证AUC从0.84540提升至0.84615，并在相当预算下优于HyFormer。消融实验表明查询生成是最大的贡献因素。打包的共享参数交叉注意力使H=8的推理延迟仅为H=1的1.89倍，使该桥接块成为高效的可堆叠统一块。
英文摘要
Post-click conversion rate (pCVR) prediction requires jointly modeling feature interactions and sequential user behaviors. The KDD Cup 2026 Tencent UniRec Challenge calls for a unified architecture addressing both. We observe that existing unified architectures often generate query tokens---the central information hub---with projection-based multi-layer perceptrons (MLPs), without explicit token-to-query attention for refining the query side. We propose QueryFormer, centered on a stackable unified field--sequence block that bridges non-sequential multi-field features and behavioral sequences, and provide a latency-aware scaling study over view width $H$, model width, depth, data, and compute. The block generates queries through cross-attention and packs sequence queries into shared-parameter attention. QueryFormer secured 1st place in the Industrial Track, achieving an official test area under the ROC curve (AUC) of 0.83254; a modest post-competition scale-up reached 0.832713. Within our grid, $H$-scaling improves validation AUC from 0.84540 to 0.84615 and beats HyFormer at comparable budgets. Ablation identifies query generation as the largest contributor. Packed shared-parameter cross-attention keeps H=8 inference latency to only 1.89x that of H=1, positioning the bridge as an efficient stackable unified block.
