KILT: a Benchmark for Knowledge Intensive Language Tasks
Facebook AI Research（脸书人工智能研究院）；University College London（伦敦大学学院）；LORIA（洛林计算机科学及其应用实验室）；University of Cambridge（剑桥大学）；HuggingFace（HuggingFace公司）；University of Amsterdam（阿姆斯特丹大学）
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vass… 展开作者
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
Challenging problems such as open-domain question answering, fact checking, slot filling and entity linking require access to large, external knowledge sources. While some models do well on individual tasks, developing general models is difficult as each task might require computationally expensive indexing of custom knowledge sources, in addition to dedicated infrastructure. To catalyze research on models that condition on specific information in large textual resources, we present a benchmark for knowledge-intensive language tasks (KILT). All tasks in KILT are grounded in the same snapshot of Wikipedia, reducing engineering turnaround through the re-use of components, as well as accelerating research into task-agnostic memory architectures. We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline, outperforming more tailor-made approaches for fact checking, open-domain question answering and dialogue, and yielding competitive results on entity linking and slot filling, by generating disambiguated text. KILT data and code are available at https://github.com/facebookresearch/KILT.
CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Amsterdam（阿姆斯特丹大学）
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus
Facebook AI Research（脸书人工智能研究院）；University College London（伦敦大学学院）；University of Mannheim（曼海姆大学）；ENS, PSL University（巴黎高等师范学院，巴黎文理研究大学）；Inria（法国国家信息与自动化研究所）；University of Washington（华盛顿大学）
Knowledge Rumination for Pre-trained Language Models
Zhejiang University（浙江大学）；Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph（浙江大学-蚂蚁集团知识图谱联合实验室）；Alibaba Group（阿里巴巴集团）；Donghai Laboratory（东海实验室）
RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoning
Wangxuan Institute of Computer Technology, Peking University（北京大学王萱计算机技术研究所）；University of Chicago（芝加哥大学）；University of California, Los Angeles（加州大学洛杉矶分校）
AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；Shanghai University of Electric Power（上海电力大学）；East China Normal University（华东师范大学）；Guangdong University of Finance and Economics（广东财经大学）；Alibaba Group（阿里巴巴集团）
KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；The University of Osaka（大阪大学）
Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answering
Interdisciplinary Program in Artificial Intelligence（人工智能交叉学科项目）；Department of Intelligence and Information（智能与信息系）；Samsung Advanced Institute of Technology（三星先进技术研究院）
On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Department of ECE University of Minnesota（电子工程系明尼苏达大学）；Data Science Institute University of Chicago（数据科学研究所芝加哥大学）；Carlson School of Management University of Minnesota（卡尔森管理学院明尼苏达大学）；Department of Surgery University of Minnesota（外科系明尼苏达大学）；Cisco Research（思科研究）；School of Statistics University of Minnesota（统计学学院明尼苏达大学）
University of Oregon Eugene OR USA；Pacific Northwest National Laboratory Richland WA USA；Adobe Research San Jose CA USA；Texas A\&M University College Station TX USA；Rensselaer Polytechnic Institute Troy NY USA；University of Oregon；Pacific Northwest National Laboratory；Adobe Research；Texas A\&M University；Rensselaer Polytechnic Institute
LSDTs: LLM-Augmented Semantic Digital Twins for Adaptive Knowledge-Intensive Infrastructure Planning
Department of Civil & Environmental Engineering, University of Maryland, College Park（大学公园马里兰大学土木与环境工程系）；Center for Urban Science and Progress, New York University（纽约大学城市科学与进步中心）；Department of Computer Science, University of Alabama（阿拉巴马大学计算机科学系）；School of Information, University of South Florida（佛罗里达州立大学信息学院）
