Conformer-Kernel with Query Term Independence for Document Retrieval

Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track

We benchmark Conformer-Kernel models under the strict blind evaluation setting of the TREC 2020 Deep Learning track. In particular, we study the impact of incorporating: (i) Explicit term matching to complement matching based on learned representations (i.e., the "Duet principle"), (ii) query term independence (i.e., the "QTI assumption") to scale the model to the full retrieval setting, and (iii) the ORCAS click data as an additional document description field. We find evidence which supports that all three aforementioned strategies can lead to improved retrieval quality.

Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence

The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark -- and can be considered to be an efficient (but slightly less effective) alternative to other Transformer-based architectures that employ (i) large-scale pretraining (high training cost), (ii) joint encoding of query and document (high inference cost), and (iii) larger number of Transformer layers (both high training and high inference costs). Since, a variant of the TK model -- called TKL -- has been developed that incorporates local self-attention to efficiently process longer…

Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension

We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder. This formulation addresses a key challenge in machine comprehension by requiring a standalone representation of the document discourse. It additionally leads to a significant scalability advantage since the encoding of the answer candidate phrases in the document can be pre-computed and indexed offline for efficient retrieval. We experiment with baseline models for the new task, which achieve a reasonable accuracy but significantly underpe…

LumberChunker：长篇叙事文档分割

LumberChunker: Long-Form Narrative Document Segmentation

提出LumberChunker方法，利用LLM动态分割文档以提升检索性能，并引入GutenQA基准验证其在RAG管道中优于传统分块方法和竞争基线。

Stochastic RAG：通过期望效用最大化的端到端检索增强生成

Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization

本文提出Stochastic RAG，将检索建模为无放回随机采样，并利用straight-through Gumbel-top-k实现端到端可微优化，在七个数据集中六个上取得最先进结果。

Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks

Classical information retrieval (IR) methods, such as query likelihood and BM25, score documents independently w.r.t. each query term, and then accumulate the scores. Assuming query term independence allows precomputing term-document scores using these models---which can be combined with specialized data structures, such as inverted index, for efficient retrieval. Deep neural IR models, in contrast, compare the whole query to the document and are, therefore, typically employed only for late stage re-ranking. We incorporate query term independence assumption into three state-of-the-art neural I…