RAG / 检索增强生成

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, Bo Zheng, Kaifu Zhang

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Citekit: A Modular Toolkit for Large Language Model Citation Generation

Jiajun Shen, Tong Zhou, Yubo Chen, Kang Liu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 7 pages, 14 figures

Attention with Dependency Parsing Augmentation for Fine-Grained Attribution

Qiang Ding, Lvzhou Luo, Yixuan Cao, Ping Luo

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, submitted to ACL ARR 2024 October

A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look

Shivani Upadhyay, Ronak Pradeep, Nandan Thakur, Daniel Campos, Nick Craswell, Ian Soboroff, Hoa Trang Dang, Jimmy Lin

专题命中 RAG评测 ：RAG(abstract)；分类 cs.IR、cs.CL

MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents

Liyan Tang, Philippe Laban, Greg Durrett

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

KodeXv0.1: A Family of State-of-the-Art Financial Large Language Models

Neel Rajani, Lilli Kiessling, Aleksandr Ogaltsov, Claus Lang

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 8 figures

Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

Sebastian Kahl, Felix Löffler, Martin Maciol, Fabian Ridder, Marius Schmitz, Jennifer Spanagel, Jens Wienkamp, Christopher Burgahn, Malte Schilling

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments The article is an extended version of a paper presented at the International Workshop on AI in Education and Educational Research (AIEER) at ECAI-2024 (27th European Conference on Artificial Intelligence)

A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case

Sonia Meyer, Shreya Singh, Bertha Tam, Christopher Ton, Angel Ren

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Multi-step Inference over Unstructured Data

Aditya Kalyanpur, Kailash Karthik Saravanakumar, Victor Barres, CJ McFate, Lori Moon, Nati Seifu, Maksim Eremeev, Jose Barrera, Abraham Bautista-Castillo, Eric Brown, David Ferrucci

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use

Yuhan Chen, Ang Lv, Ting-En Lin, Changyu Chen, Yuchuan Wu, Fei Huang, Yongbin Li, Rui Yan

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments ACL 2024 main

Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs

Fatemeh Shiri, Van Nguyen, Farhad Moghimifar, John Yoo, Gholamreza Haffari, Yuan-Fang Li

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation

Gauthier Guinet, Behrooz Omidvar-Tehrani, Anoop Deoras, Laurent Callot

专题命中 RAG评测 ：RAG(abstract)；分类 cs.IR、cs.CL

Comments Proceedings of the 41st International Conference on Machine Learning (ICML), 29 pages, 12 figures

Can large language models understand uncommon meanings of common words?

Jinyang Wu, Feihu Che, Xinxin Zheng, Shuai Zhang, Ruihan Jin, Shuai Nie, Pengpeng Shao, Jianhua Tao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification

Haoqiang Kang, Juntong Ni, Huaxiu Yao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss

Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 11M tokens, fix qa3 min facts per task in Table 1

Experimenting with Large Language Models and vector embeddings in NASA SciX

Sergi Blanco-Cuaresma, Ioana Ciucă, Alberto Accomazzi, Michael J. Kurtz, Edwin A. Henneken, Kelly E. Lockhart, Felix Grezes, Thomas Allen, Golnaz Shapurian, Carolyn S. Grant, Donna M. Thompson, Timothy W. Hostetler, Matthew R. Templeton, Shinyi Chen, Jennifer Koch, Taylor Jacovich, Daniel Chivvis, Fernanda de Macedo Alves, Jean-Claude Paquin, Jennifer Bartlett, Mugdha Polimera, Stephanie Jarmak

专题命中 RAG评测 ：retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments To appear in the proceedings of the 33th annual international Astronomical Data Analysis Software & Systems (ADASS XXXIII)

Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination

Haoqiang Kang, Xiao-Yang Liu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Chainpoll: A high efficacy method for LLM hallucination detection

Robert Friel, Atindriyo Sanyal

专题命中 RAG评测 ：retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

A Study of Situational Reasoning for Traffic Understanding

Jiarui Zhang, Filip Ilievski, Kaixin Ma, Aravinda Kollaa, Jonathan Francis, Alessandro Oltramari

专题命中 RAG评测 ：dense retrieval(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 6 figures, 5 tables, camera ready version of SIGKDD 2023

CAIA in Practice: Field Evaluation of an AI-Assisted Support System for Text-Based Online Counselling

CAIA实践：AI辅助文本在线咨询支持系统的实地评估

Philipp Steigerwald, Nico Bienlein, Jennifer Burghardt, Mara Stieler, Robert Lehmann, Jens Albrecht

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL

AI总结 该研究通过实地评估验证了协同设计的AI辅助文本在线咨询工具CAIA，其7个LLM驱动功能经检索增强生成优化，获34名专业咨询师广泛采用，核心是保障专业自主权与信息准确性，助力心理咨询实践。

Comments Accepted at IEEE ICTAI 2025

Journal ref 2025 IEEE 37th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 1476-1483

ZenGen: Social Mind for LLMs

Zing：大语言模型的社交智能

ZenGen Team, Ao Xiang, Bi Jingping, Chen Jiahui, Chen Lehan, Chen Yilin, Cheng Xueqi, Fan Yixing, Gan Kairong, Gao Haowen, Gao Jinhua, Gao Shuxuan, Gong Chang, Guo Jiafeng, Guo Ruijie, Han Zhouyu, He Guangfu, He Yichun, Jiang Shuo, Jing Shaoling, Jing Ya, Lei Chenhao, Lei Yan, Li Anqi, Li Chengao, Li Haoyu, Li Shitian, Liang Xinjian, Liu Zhaoge, Lyu Xingyu, Nie Zhuwei, Pang Liang, Quan Zeping, Shan Shiguang, Shen Huawei, Tang Xinran, Tian Feng, Wang Qian, Wang Ruiping, Wang Xiaohong, Xia Zaiyu, Xiao Yi, Xu Jiayuan, Xu Kehan, Xu Qianqian, Xu Tianyu, Xu Yongjun, Yang Haoming, Yang Jun, Yao Di, Yu Xiaoming, Zhang Futong, Zhang Jie, Zhang Shixuan, Zhang Yuxuan, Zhao Xinyu, Zhao Zhuoran, Zhong Yunfei, Zhu Shengyu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL

AI总结 研究大语言模型社交智能，提出Zijing框架，含测量基准SoMBench、训练方法Zing及推理架构Actio，通过实验证明社交智能大语言模型在评估、内化和落地方面需协同发展。

ContextSniper: AntTrail's Token-Efficient Code Memory for Repository-Level Program Repair

ContextSniper: AntTrail的令牌高效代码记忆用于仓库级程序修复

Chiwang Luk, Matin Mohammad Najafi, Zhifeng Jia, Wei Yang, Xiuchang Li, Jinwei Zhu, Yang Ren, Lei Chen, Gao Cong

机构 * Huawei（华为） ； HKUST(GZ)（香港科技大学（广州）） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 RAG评测 ：RAG(abstract)；分类 cs.AI

AI总结 提出ContextSniper，一种令牌高效的代码记忆层，通过精准证据选择、混合检索排序和意图感知上下文门控，在SWE-bench Lite上减少51.5%令牌使用和36.4%成本，修复率仅轻微下降。

Eval4Sim: An Evaluation Framework for Persona Simulation

Eval4Sim: 一种用于人设模拟的评估框架

Eliseo Bao, Anxo Perez, Javier Parapar, Xi Wang

机构 * University of Sheffield（谢菲尔德大学） 纠错

专题命中 RAG评测 ：dense retrieval(abstract)；分类 cs.CL

AI总结 Eval4Sim提出了一种评估框架，用于衡量模拟对话与人类对话模式的契合度，通过三个互补维度评估人设的忠实性、一致性和自然性。

Comments Accepted at CIKM 2026

When Is Complex Chunking Worth It? A Multi-Objective Evaluation of Chunking Methods at Scale

复杂分块何时值得？大规模分块方法的多目标评估

Laura Caspari, Kanishka Ghosh Dastidar, Michael Dinzinger, Jelena Mitrović, Michael Granitzer

专题命中 RAG评测 ：dense retrieval(abstract)；分类 cs.IR

AI总结 该研究针对长文档分块的多目标评估问题，在多语料库、多模型下对比八种分块策略，发现分块需兼顾检索性能与系统成本，应作为多目标设计决策。

TopoIntent: Compiling Security Intent into Executable, Compliance-Checked Network Topologies

TopoIntent：将安全意图编译为可执行、合规性检查的网络拓扑结构

Xiaokang Qu, Jianliang Ma, Zao Fan, Tianshu Chu, Tianlong Fan, Linyuan Lü

机构 * University of Science and Technology of China（中国科学技术大学） ； Topsec Technologies Group Inc.（天融信科技集团股份有限公司） 纠错

专题命中 RAG评测 ：vector search(abstract)；分类 cs.AI

AI总结 TopoIntent系统通过模式契约、向量搜索和分阶段融合将安全意图编译为合规可执行拓扑，经评估其修复后CIS满意度与ACL通过率均显著提升。

Citation Grounding Measures the Oracle: Graph Coverage Determines Reported LLM Hallucination Rates in Law

引用溯源：通过法律引用图检测和减少LLM引用幻觉

Volodymyr Ovcharov

机构 * LEX AI LLC 纠错

专题命中 RAG评测 ：RAG(abstract_cn)；分类 cs.CL

AI总结 提出引用溯源（CG）指标，利用乌克兰法院判决的引用图（1.008亿判决，5.02亿边）检测LLM法律引用幻觉，并通过CG-DPO方法（基于真实判决构建偏好对）减少幻觉，在100个法律查询上CG为0.791-0.873，幻觉率13-21%。

Comments 21 pages, 4 figures, 5 tables. Substantially revised: title, framing and several v1 results changed. Adds a coverage sweep and a separability analysis; corrects the DPO configuration, the density-accuracy correlation and the qualitative examples. Code and data: https://huggingface.co/datasets/overthelex/citation-grounding-eval

DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction

DiagChain：用于评估大语言模型智能体基于证据的攻击链重构的诊断基准

Xuyang Liu, Yibin Han, Zhenwei Zhang, Kai Chang, Zhiwei Xu, Tian Qiu, Weixian Deng, Jiabao Gao, Xiaolin Peng, Hai Wan, Xibin Zhao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.AI

AI总结 研究针对LLM智能体攻击链重构基准的不足，提出DiagChain基准与ECRAG方法，经6种LLM评估发现模型在证据整合与排序环节存在瓶颈，为网络安全智能体改进提供了诊断依据。

Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education

Teachy Mini：用于高等教育的基于知识的生成式社交机器人的开发与初步评估

Stephan Vonschallen, Karim Kaufmann, Dominique Oberle, Friederike Eyssel, Theresa Schmiedel

机构 * Institute of Information Systems, Zurich University of Applied Sciences（苏黎世应用科学大学信息系统研究所） ； Center for Cognitive Interaction Technology, Bielefeld University（比勒费尔德大学认知交互技术中心） 纠错

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.AI

AI总结 研究针对生成式社交机器人在高等教育辅导中的风险，基于知识的设计要求开发Teachy Mini系统，经初步评估，该系统在负责任辅导行为及个性化等方面表现更佳，虽在部分方面与对照无显著差异，但显示出基于知识的设计对学习效果有积极作用。

CAFE: A Compound-AI Factorial Evaluation Framework

CAFE：一种复合人工智能因子评估框架

Fabian Lukassen, Christoph Weisser, Thomas Kneib, Alexander Silbersdorff

机构 * University of Göttingen（哥廷根大学） ； Bielefeld University of Applied Sciences and Arts (HSBI)（比勒费尔德应用科学与艺术大学） 纠错

专题命中 RAG评测 ：retriever(abstract)；分类 cs.CL

AI总结 CAFE是开源平台，用于复合人工智能系统评估。它将管道组件设为因子构建析因设计，运行配置并用模型评判器和人工评分。能归因质量差异，报告多种结果，还能解释组件影响，在问答管道验证有效，已发布为Python包和Web应用。

CAGE-1: Control, Assurance, and Governance Evaluation for Enterprise Agentic AI

CAGE-1：企业智能代理人工智能的控制、保证和治理评估

Roopam W. Sure

机构 * Independent technical report（独立技术报告） 纠错

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.AI

AI总结 研究企业人工智能从实验转向运营工作流带来的评估问题，介绍CAGE-1评估框架，评估多方面内容，引入预绑定保证，用于判断企业代理是否可部署。

Comments 17 pages, 3 figures; independent technical report