Retrieval-Augmented Generation (RAG) has proven to be an effective method for mitigating hallucination issues inherent in large language models (LLMs). Previous approaches typically train retrievers based on semantic similarity, lacking optimization for RAG. More recent works have proposed aligning retrievers with the preference signals of LLMs. However, these preference signals are often difficult for dense retrievers, which typically have weaker language capabilities, to understand and learn effectively. Drawing inspiration from pedagogical theories like Guided Discovery Learning, we propose a novel framework, FiGRet (Fine-grained Guidance for Retrievers), which leverages the language capabilities of LLMs to construct examples from a more granular, information-centric perspective to guide the learning of retrievers. Specifically, our method utilizes LLMs to construct easy-to-understand examples from samples where the retriever performs poorly, focusing on three learning objectives highly relevant to the RAG scenario: relevance, comprehensiveness, and purity. These examples serve as scaffolding to ultimately align the retriever with the LLM's preferences. Furthermore, we employ a dual curriculum learning strategy and leverage the reciprocal feedback between LLM and retriever to further enhance the performance of the RAG system. A series of experiments demonstrate that our proposed framework enhances the performance of RAG systems equipped with different retrievers and is applicable to various LLMs.
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance
Department of Computer Science and Technology, Northeastern University, China（东北大学计算机科学与技术系）；Department of Computer Science and Technology, Institute for AI, Tsinghua University, China（清华大学人工智能研究院计算机科学与技术系）；Beijing National Research Center for Information Science and Technology, China（北京信息科学与技术国家研究中心）
Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting
Computer Aided Medical Procedures, Technische Universität München, Germany（计算机辅助医疗程序，慕尼黑技术大学，德国）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心，德国）
A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation
School of Physics and Information Technology, Shaanxi Normal University（陕西师范大学物理与信息技术学院）；Department of Computing Technologies, Swinburne University of Technology（斯威本科技大学计算技术系）；Department of Software Engineering, University of Szeged（塞格德大学软件工程系）
MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Alibaba Cloud Computing（阿里云计算）；Hong Kong University of Science and Technology（香港科技大学）
RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；MThreads, Inc.（MThreads公司）；Nanyang Technological University（南洋理工大学）
CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG
School of Computer Science and Engineering, UESTC, Chengdu, China（电子科技大学计算机科学与工程学院，成都，中国）；IDEAS Lab, University of Birmingham, Birmingham, UK（伯明翰大学IDEAS实验室，英国布里斯托尔）
University of Technology Sydney（悉尼科技大学）；Microsoft Corporation（微软公司）
Long document retrieval aims to fetch query-relevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a heterogeneous yet powerful cross-encoder. However, in contrast to passages or sentences, retrieval on long documents suffers from the scope hypothesis that a long document may cover multiple topics. This maximizes their structure heterogeneity and poses a granular-mismatch issue, leading to an inferior distillation efficacy. In this work, we propose a new learning framework, fine-grained distillation (FGD), for long…
Understanding the Behaviors of Environment-aware Information Retrieval
Fudan University（复旦大学）；Alibaba DAMO Academy（阿里巴巴达摩院）；Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）；Shanghai University of Finance and Economics（上海财经大学）
Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions b…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
We present a new scientific document similarity model based on matching fine-grained aspects of texts. To train our model, we exploit a naturally-occurring source of supervision: sentences in the full-text of papers that cite multiple papers together (co-citations). Such co-citations not only reflect close paper relatedness, but also provide textual descriptions of how the co-cited papers are related. This novel form of textual supervision is used for learning to match aspects across papers. We develop multi-vector representations where vectors correspond to sentence-level aspects of documents…
FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval
Shandong University（山东大学）；City University of Hong Kong（香港城市大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Shandong Jianzhu University（山东建筑大学）
ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), TU Darmstadt（图腾知识处理实验室（UKP实验室）、计算机科学系和海德堡人工智能中心（hessian.AI）、图腾达姆斯塔特大学）；Hebrew University of Jerusalem and The Allen Institute for AI (AI2)（耶路撒冷希伯来大学和人工智能研究所（AI2））
Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval
Shanghai Jiao Tong University（上海交通大学）；Institute of Digital Twin, Eastern Institute of Technology（数字孪生研究院，东部技术研究所）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）
