RAG / 检索增强生成

MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems

Nandan Thakur, Suleman Kazi, Ge Luo, Jimmy Lin, Amin Ahmad

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted at NAACL 2025 (Main Conference)

ClaimTrust: Propagation Trust Scoring for RAG Systems

Hangkai Qian, Bo Li, Qichen Wang

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL

Comments 6 pages, 2 figures, 1 table

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL

Comments https://github.com/OpenBMB/RAGEval

LettuceDetect: A Hallucination Detection Framework for RAG Applications

Ádám Kovács, Gábor Recski

专题命中 RAG评测 ：RAG(title,abstract)；retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments 6 pages

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models

Aryan Jadon, Avinash Patil, Shashank Kumar

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 8 Pages

WASHtsApp -- A RAG-powered WhatsApp Chatbot for supporting rural African clean water access, sanitation and hygiene

Simon Kloker, Alex Cedric Luyima, Matthew Bazanya

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.AI

Comments Working Paper. Accepted at IST-Africa Conference 2025, Nairobi

Evaluation of Retrieval-Augmented Generation: A Survey

Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, Zhaofeng Liu

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Better RAG using Relevant Information Gain

Marc Pickett, Jeremy Hartman, Ayan Kumar Bhowmick, Raquib-ul Alam, Aditya Vempaty

专题命中 RAG评测 ：RAG(title,abstract)；retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments 4 page paper submitted to EMNLP

Generating Diverse Q&A Benchmarks for RAG Evaluation with DataMorgana

Simone Filice, Guy Horowitz, David Carmel, Zohar Karnin, Liane Lewin-Eytan, Yoelle Maarek

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL

Federated Learning and RAG Integration: A Scalable Approach for Medical Large Language Models

Jincheol Jung, Hongju Jeong, Eui-Nam Huh

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems

Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, Marina Danilevsky

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization

Jiayi Wu, Hengyi Cai, Lingyong Yan, Hao Sun, Xiang Li, Shuaiqiang Wang, Dawei Yin, Ming Gao

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation

Hyeonseok Lim, Dongjae Shin, Seohyun Song, Inho Won, Minjun Kim, Junghun Yuk, Haneol Jang, KyungTae Lim

专题命中 RAG评测 ：retrieval augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments The 31st International Conference on Computational Linguistics (COLING 2025), 19 pages

Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation

Jirui Qi, Gabriele Sarti, Raquel Fernández, Arianna Bisazza

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024 Main Conference. Code and data released at https://github.com/Betswish/MIRAGE

Journal ref Proceedings of EMNLP (2024) 6037-6053

Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework

Ronak Pradeep, Nandan Thakur, Shivani Upadhyay, Daniel Campos, Nick Craswell, Jimmy Lin

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL

UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis

Yulong Hui, Yao Lu, Huanchen Zhang

专题命中 RAG评测 ：retrieval augmented generation(title)；retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarios

Hai Lin, Shaoxiong Zhan, Junyou Su, Haitao Zheng, Hui Wang

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.IR、cs.AI

SFR-RAG: Towards Contextually Faithful LLMs

Xuan-Phi Nguyen, Shrey Pandit, Senthil Purushwalkam, Austin Xu, Hailin Chen, Yifei Ming, Zixuan Ke, Silvio Savarese, Caiming Xong, Shafiq Joty

专题命中 RAG评测 ：RAG(title,abstract)；retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments Technical report

RAGSys: Item-Cold-Start Recommender as RAG System

Emile Contal, Garrin McGoldrick

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL

NinjaLLM: Fast, Scalable and Cost-effective RAG using Amazon SageMaker and AWS Trainium and Inferentia2

Tengfei Xue, Xuefeng Li, Roman Smirnov, Tahir Azim, Arash Sadrieh, Babak Pahlavan

专题命中 RAG评测 ：RAG(title,abstract)；retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese

Yunqi Xu, Tianchi Cai, Jiyan Jiang, Xierui Song

专题命中 RAG评测 ：retrieval augmented generation(title,abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Journal ref KDD 2024 (oral)

DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation

Shuting Wang, Jiongnan Liu, Shiren Song, Jiehan Cheng, Yuqi Fu, Peidong Guo, Kun Fang, Yutao Zhu, Zhicheng Dou

专题命中 RAG评测 ：retrieval-augmented generation(title,abstract)；RAG(abstract)；分类 cs.IR、cs.CL

Evaluating Students' Open-ended Written Responses with LLMs: Using the RAG Framework for GPT-3.5, GPT-4, Claude-3, and Mistral-Large

Jussi S. Jauhiainen, Agustín Garagorry Guerra

专题命中 RAG评测 ：RAG(title,abstract)；retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 6 tables, 1 figure

Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup

Tristan Kenneweg, Philip Kenneweg, Barbara Hammer

专题命中 RAG评测 ：retrieval augmented generation(title,abstract)；RAG(abstract)；分类 cs.IR、cs.CL

Comments Was handed in to IJCNN prior to preprint publication here. Was neither accepted nor rejected at date of publication here

Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations

结合标题链前缀的结构感知语义分块：1600次查询评估及文本变换消融实验中的测量陷阱

Yang Yang

专题命中 RAG评测 ：RAG(summary_cn,abstract)；retrieval-augmented generation(abstract)

AI总结 本文提出仅在分块侧的三阶段语义分块流水线，经1600次查询评估提升RAG的MRR@5，还发现分块研究中存在的测量陷阱并推荐检索时每个候选前缀评估的方向。

Comments 8 pages, 1 table. Replication package: DOI 10.5281/zenodo.21744653

RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora

RARE：面向高相似性语料的冗余感知检索评估框架

Hanjun Cho, Jay-Yoon Lee

机构 * Allganize ； Seoul National University（首尔国立大学） 纠错

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；retrieval-augmented generation(abstract)；retriever(abstract)；分类 cs.IR、cs.CL、cs.AI

AI总结 RARE通过分解文档为原子事实和增强LLM生成数据，构建更真实的基准，揭示当前评估体系无法捕捉的鲁棒性差距。

Comments Accepted to ACL 2026 (Main Conference)

Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents

记忆决定差异：评估不同记忆角色如何塑造对话代理

Yuxin Wang, Paul Thomas, Zhiwei Yu, Yuan Gao, Saeed Hassanpour, Soroush Vosoughi, Robert Sim, Nick Craswell

机构 * Dartmouth College（达特茅斯学院） ； Microsoft（微软公司） 纠错

专题命中 RAG评测 ：RAG(summary_cn,abstract)；分类 cs.IR、cs.CL、cs.AI

AI总结 本研究提出对话记忆细粒度分类法，通过用户中心评估框架，揭示不同类型记忆（如澄清记忆、无关记忆）对RAG对话系统响应准确性、个性化及主题相关性的差异化影响。

Train, Retrieve, or Both? A Four-Arm Head-to-Head for Correct Statutory Citation on the Ontario Residential Tenancies Act

训练、检索，还是两者兼用？针对安大略省住宅租赁法的正确法定引用的四组头对头比较

Ali Asaria, Tony Salomone, Deep Gandhi

机构 * Transformer Lab 纠错

专题命中 RAG评测 ：RAG(summary_cn,abstract)；hybrid retrieval(abstract)

AI总结 研究自诉租户、房东和帮助台工作人员如何获得正确的法定引用，通过四组实验比较微调、检索及混合方法，发现SFT+RAG混合模型在精确匹配上得分最高且无幻觉引用。

Ablation Study of a Fairness Auditing Agentic System for Bias Mitigation in Early-Onset Colorectal Cancer Detection

公平性审计代理系统在早发性结直肠癌检测中缓解偏见的消融研究

Amalia Ionescu, Jose Guadalupe Hernandez, Jui-Hsuan Chang, Emily F. Wong, Paul Wang, Jason H. Moore, Tiffani J. Bright

专题命中 RAG评测 ：RAG(summary_cn,abstract)；retrieval-augmented generation(abstract)

AI总结 提出双代理架构（领域专家代理+公平性顾问代理），通过消融实验比较不同配置下大语言模型在公平性审计中的表现，发现带RAG的代理系统在识别差异方面语义相似度最高。

Tutor, Not Solver: Designing a Guardrailed AI Assistant for Learning in Higher Education: A Design Case of PeteChat

导师，而非解题者：设计高等教育中带护栏的AI学习助手——PeteChat的设计案例

Belle Li, Lily Tan, Wei Zakharov, Qiang Qiu, Colby Ben Acton

专题命中 RAG评测 ：RAG(summary_cn,abstract)；retrieval-augmented generation(abstract)

AI总结 本文通过设计案例PeteChat，提出八项可迁移的评估感知AI导师设计原则，包括作业护栏、调试支架等，基于本地Llama-3模型和RAG技术，旨在平衡学习支持与学术诚信。

Comments Preprint. Includes supplementary appendices, interface figures, and baseline-analysis tables