Recently, there have been significant advances in neural methods for tackling knowledge-intensive tasks such as open domain question answering (QA). These advances are fueled by combining large pre-trained language models with learnable retrieval of documents. Majority of these models use separate encoders for learning query representation, passage representation for the retriever and an additional encoder for the downstream task. Using separate encoders for each stage/task occupies a lot of memory and makes it difficult to scale to a large number of tasks. In this paper, we propose a novel Retrieval Optimized Multi-task (ROM) framework for jointly training self-supervised tasks, knowledge retrieval, and extractive question answering. Our ROM approach presents a unified and generalizable framework that enables scaling efficiently to multiple tasks, varying levels of supervision, and optimization choices such as different learning schedules without changing the model architecture. It also provides the flexibility of changing the encoders without changing the architecture of the system. Using our framework, we achieve comparable or better performance than recent methods on QA, while drastically reducing the number of parameters.
GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Framework
School of Future Science and Engineering, Soochow University(未来科学与工程学院,苏州大学);School of Mathematical Sciences, Soochow University(数学学院,苏州大学);School of Information Technology (Smart Campus Education Center), Suzhou Institute of Trade & Commerce(信息技术学院(智能校园教育中心),苏州贸易与商业学院)
Optimizing Legal Document Retrieval in Vietnamese with Semi-Hard Negative Mining
Faculty of Computer Science, University of Information Technology, Ho Chi Minh City, Vietnam(信息科技学院计算机科学系,胡志明市,越南);Faculty of Information Science and Engineering, University of Information Technology, Ho Chi Minh City, Vietnam(信息科技学院信息科学与工程系,胡志明市,越南);Laboratory of Multimedia Communications, University of Information Technology, Ho Chi Minh City, Vietnam(信息科技学院多媒体通信实验室,胡志明市,越南);Vietnam National University, Ho Chi Minh City, Vietnam(越南国家大学,胡志明市,越南)
Traditional Chinese Medicine Case Analysis System for High-Level Semantic Abstraction: Optimized with Prompt and RAG
Chongqing Jiudun Technology Co., Ltd.(重庆九盾科技有限公司);School of Mechanical Science & Engineering, Huazhong University of Science and Technology(华中科技大学机械科学与工程学院);Xinjiang Medical University(新疆医科大学);School of Nursing, Shanghai Jiao Tong University(上海交通大学护理学院);School of Mathematical Sciences, Shanghai Jiao Tong University(上海交通大学数学科学学院)
LLM-Specific Utility for Retrieval-Augmented Generation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Baidu Inc(百度公司)
Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
Meta;Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(Inria 巴黎-罗克quencourt 分所);Rajiv Gandhi University(拉朱·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室)
APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation
University of Chinese Academy of Sciences(中国科学院大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部);Wenge Technology Co., Ltd(Wenger科技有限公司)
Topic-DPR: Topic-based Prompts for Dense Passage Retrieval
South China Normal University(华南师范大学);The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
LogosKG: Hardware-Optimized Scalable and Interpretable Knowledge Graph Retrieval
LARK Lab, University of Colorado Anschutz(洛克拉克实验室,科罗拉多大学安施图茨分校);University of Colorado Boulder(科罗拉多大学波德分校);Loyola University Chicago(芝加哥洛克拉克大学);Harvard Medical School(哈佛医学院);Boston Children’s Hospital(波士顿儿童医院);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval
Beijing University of Posts and Telecommunications(北京邮电大学);Microsoft Research Asia(微软亚洲研究院);Microsoft STCA(微软STCA);University of Science and Technology of China(中国科学技术大学)