基于解耦异或注意力的链接式用户-物品交互建模
LIME: Link-based User-item Interaction Modeling with Decoupled XOR Attention for Efficient Test Time Scaling
AI总结：
LIME通过解耦异或注意力机制，实现了高效推荐系统中计算复杂度的降低，提升了推理速度和用户参与度。
摘要：
扩展大型推荐系统需要推进三个主要前沿：处理更长的用户历史、扩大候选集和增加模型容量。尽管有前景，但transformers的计算成本随着用户序列长度呈二次增长，随着候选集数量呈线性增长。这种权衡使得在推理时扩展候选集或增加序列长度变得成本高昂，尽管这能显著提升性能。我们引入了LIME，一种新的架构，解决了这一权衡。通过两个关键创新，LIME从根本上降低了计算复杂性。首先，低秩"链接嵌入"使通过解耦用户和候选交互来预计算注意力权重成为可能，使推理成本几乎与候选集大小无关。其次，一种线性注意力机制，LIME-XOR，将计算复杂性从二次（O(N²)）降低到线性（O(N)）。在公开和工业数据集上的实验表明，LIME在与最先进transformers接近的同时，对大候选集或长序列长度的推理速度提升了10倍。当在主要推荐平台上测试时，LIME在保持候选集大小和用户历史长度的最小推理成本的同时，提高了用户参与度，建立了高效且表达能力强的推荐系统的新范式。
英文摘要：
Scaling large recommendation systems requires advancing three major frontiers: processing longer user histories, expanding candidate sets, and increasing model capacity. While promising, transformers' computational cost scales quadratically with the user sequence length and linearly with the number of candidates. This trade-off makes it prohibitively expensive to expand candidate sets or increase sequence length at inference, despite the significant performance improvements. We introduce \textbf{LIME}, a novel architecture that resolves this trade-off. Through two key innovations, LIME fundamentally reduces computational complexity. First, low-rank ``link embeddings" enable pre-computation of attention weights by decoupling user and candidate interactions, making the inference cost nearly independent of candidate set size. Second, a linear attention mechanism, \textbf{LIME-XOR}, reduces the complexity with respect to user sequence length from quadratic ($O(N^2)$) to linear ($O(N)$). Experiments on public and industrial datasets show LIME achieves near-parity with state-of-the-art transformers but with a 10$\times$ inference speedup on large candidate sets or long sequence lengths. When tested on a major recommendation platform, LIME improved user engagement while maintaining minimal inference costs with respect to candidate set size and user history length, establishing a new paradigm for efficient and expressive recommendation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
