GLOW：用于Web搜索的全局加权自注意网络
GLOW : Global Weighted Self-Attention Network for Web Search
摘要：
深度匹配模型的目标是通过在第一阶段检索中将查询和文档映射到语义向量中，以便于搜索引擎检索更多相关文档。当利用BERT作为深度匹配模型时，两个词的注意力分数仅建立在本地上下文化的词嵌入上。在信息检索任务中，关键词的重要性是一个关键问题，而关键词的重要性是一个全局性的先验知识。除此之外，BERT仅在子词标记上执行注意力，这削弱了整个词的注意力表示。提出了一种新的全局加权自注意（GLOW）网络用于Web文档搜索。GLOW将全局语料统计融合到深度匹配模型中。通过将先验权重添加到来自全局信息的注意力生成中，如BM 25，GLOW成功地与查询矩阵$Q$和关键矩阵$K$一起学习加权注意力分数。我们还提出了一个有效的全词权重共享解决方案，使以前的全词知识到子词级的注意。它帮助Transformer学习全词水平的注意力。为了使我们的模型适用于复杂的网络搜索场景，我们引入了组合字段表示，以适应多个字段的文档，即使有可变数量的实例。我们证明GLOW是更有效地捕捉查询和文档中的主题和语义表示。在公共数据集上进行的内在评价和实验表明，GLOW是一个通用的文档检索任务框架。它显著优于BERT和其他竞争基线，同时保持与BERT相同的模型复杂性。
英文摘要：
Deep matching models aim to facilitate search engines retrieving more relevant documents by mapping queries and documents into semantic vectors in the first-stage retrieval. When leveraging BERT as the deep matching model, the attention score across two words are solely built upon local contextualized word embeddings. It lacks prior global knowledge to distinguish the importance of different words, which has been proved to play a critical role in information retrieval tasks. In addition to this, BERT only performs attention across sub-words tokens which weakens whole word attention representation. We propose a novel Global Weighted Self-Attention (GLOW) network for web document search. GLOW fuses global corpus statistics into the deep matching model. By adding prior weights into attention generation from global information, like BM25, GLOW successfully learns weighted attention scores jointly with query matrix $Q$ and key matrix $K$. We also present an efficient whole word weight sharing solution to bring prior whole word knowledge into sub-words level attention. It aids Transformer to learn whole word level attention. To make our models applicable to complicated web search scenarios, we introduce combined fields representation to accommodate documents with multiple fields even with variable number of instances. We demonstrate GLOW is more efficient to capture the topical and semantic representation both in queries and documents. Intrinsic evaluation and experiments conducted on public data sets reveal GLOW to be a general framework for document retrieve task. It significantly outperforms BERT and other competitive baselines by a large margin while retaining the same model complexity with BERT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
