Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance"? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.
$\texttt{MixGR}$: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity
Technical University of Darmstadt(达姆施塔特工业大学);Carnegie Mellon University(卡内基梅隆大学);University of Washington(华盛顿大学);University of Pennsylvania(宾夕法尼亚大学);Tencent AI Lab(腾讯AI实验室)
Telco-DPR: A Hybrid Dataset for Evaluating Retrieval Models of 3GPP Technical Specifications
Instituto de Telecomunicações(电信研究所);CELFINET - A Cyient Company(CELFINET - 赛文特公司);Instituto Superior de Engenharia Lisboa(里斯本高等工程学院);Instituto Superior Técnico(里斯本高等技术学院)
Your Dense Retriever is Secretly an Expeditious Reasoner
School of Computer Science and Engineering, Beihang University, China(北京航空航天大学计算机科学与工程学院);Beijing Institute for General Artificial Intelligence, China(北京通用人工智能研究院)