RAG / 检索增强生成

Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis

Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

VERA: Validation and Evaluation of Retrieval-Augmented Systems

Tianyu Ding, Adi Banerjee, Laurent Mombaerts, Yunhong Li, Tarik Borogovac, Juan Pablo De la Cruz Weinstein

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments Accepted in Workshop on Evaluation and Trustworthiness of Generative AI Models, KDD 2024

The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?

Pedro Ramoneda, Emilia Parada-Cabaleiro, Benno Weck, Xavier Serra

专题命中 RAG评测 ：retrieval augmented generation(abstract)；retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Evaluating ChatGPT on Nuclear Domain-Specific Data

Muhammad Anwar, Mischa de Costa, Issam Hammad, Daniel Lau

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Journal ref 43rd Annual CNS Conference and the 48th Annual CNS/CNA Student Conference Sheraton Cavalier Saskatoon Hotel, Saskatoon, SK, Canada, June 16-19, 2024

Plan with Code: Comparing approaches for robust NL to DSL generation

Nastaran Bassamzadeh, Chhaya Methani

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 1 figure, 5 tables. arXiv admin note: substantial text overlap with arXiv:2407.02742

Lynx: An Open Source Hallucination Evaluation Model

Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, Rebecca Qian

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networks

Pranshav Gajjar, Vijay K. Shah

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions

Bojana Bašaragin, Adela Ljajić, Darija Medvecki, Lorenzo Cassano, Miloš Košprdić, Nikola Milošević

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted at BioNLP Workshop 2024, colocated with ACL 2024

BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science

Xinna Lin, Siqi Ma, Junjie Shan, Xiaojing Zhang, Shell Xu Hu, Tiannan Guo, Stan Z. Li, Kaicheng Yu

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Generation of Asset Administration Shell with Large Language Model Agents: Toward Semantic Interoperability in Digital Twins in the Context of Industry 4.0

Yuchen Xia, Zhewen Xiao, Nasser Jazdi, Michael Weyrich

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments Published in IEEE Access

WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia

Yufang Hou, Alessandra Pascale, Javier Carnerero-Cano, Tigran Tchrakian, Radu Marinescu, Elizabeth Daly, Inkit Padhi, Prasanna Sattigeri

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

FinBen: A Holistic Financial Benchmark for Large Language Models

Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiyuan Yao, Haohang Li, Yangyang Yu, Gang Hu, Jiajia Huang, Xiao-Yang Liu, Alejandro Lopez-Lira, Benyou Wang, Yanzhao Lai, Hao Wang, Min Peng, Sophia Ananiadou, Jimin Huang

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 11 figures

Designing an Evaluation Framework for Large Language Models in Astronomy Research

John F. Wu, Alina Hyk, Kiera McCormick, Christine Ye, Simone Astarita, Elina Baral, Jo Ciuca, Jesse Cranney, Anjalie Field, Kartheik Iyer, Philipp Koehn, Jenn Kotler, Sandor Kruk, Michelle Ntampaka, Charles O'Neill, Joshua E. G. Peek, Sanjib Sharma, Mikaeel Yunus

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments 7 pages, 3 figures. Code available at https://github.com/jsalt2024-evaluating-llms-for-astronomy/astro-arxiv-bot

Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variability

Jinwei Lu, Yuanfeng Song, Haodi Zhang, Chen Zhang, Raymond Chi-Wing Wong

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

IITK at SemEval-2024 Task 2: Exploring the Capabilities of LLMs for Safe Biomedical Natural Language Inference for Clinical Trials

Shreyasi Mandal, Ashutosh Modi

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted at SemEval 2024, NAACL 2024; 8 Pages

FACTOID: FACtual enTailment fOr hallucInation Detection

Vipula Rawte, S. M Towhidul Islam Tonmoy, Krishnav Rajbangshi, Shravani Nag, Aman Chadha, Amit P. Sheth, Amitava Das

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Evaluating Very Long-Term Conversational Memory of LLM Agents

Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, Yuwei Fang

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 19 pages; Project page: https://snap-research.github.io/locomo/

Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems

Shengzhe Xu, Christo Kurisummoottil Thomas, Omar Hashash, Nikhil Muralidhar, Walid Saad, Naren Ramakrishnan

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

How well do LLMs cite relevant medical references? An evaluation framework and analyses

Kevin Wu, Eric Wu, Ally Cassasola, Angela Zhang, Kevin Wei, Teresa Nguyen, Sith Riantawan, Patricia Shi Riantawan, Daniel E. Ho, James Zou

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately

Liang Zhang, Katherine Jijo, Spurthi Setty, Eden Chung, Fatima Javid, Natan Vidra, Tommy Clifford

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Weaver: Foundation Models for Creative Writing

Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, Yibin Liu, Jialong Wu, Shengwei Ding, Long Li, Zhiwei Huang, Xinle Deng, Teng Yu, Gangan Ma, Han Xiao, Zixin Chen, Danjun Xiang, Yunxia Wang, Yuanyuan Zhu, Yi Xiao, Jing Wang, Yiru Wang, Siran Ding, Jiayang Huang, Jiayi Xu, Yilihamu Tayier, Zhenyu Hu, Yuan Gao, Chengfeng Zheng, Yueshu Ye, Yihang Li, Lei Wan, Xinyue Jiang, Yujie Wang, Siyu Cheng, Zhule Song, Xiangru Tang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, Yuchen Eleanor Jiang, Wangchunshu Zhou

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Evaluating and Enhancing Large Language Models Performance in Domain-specific Medicine: Osteoarthritis Management with DocOA

Xi Chen, MingKe You, Li Wang, WeiZhi Liu, Yu Fu, Jie Xu, Shaoting Zhang, Gang Chen, Kang Li, Jian Li

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 16 Pages, 7 Figures

Minimizing Factual Inconsistency and Hallucination in Large Language Models

Muneeswaran I, Shreya Saxena, Siva Prasad, M V Sai Prakash, Advaith Shankar, Varun V, Vishal Vaddina, Saisubramaniam Gopalakrishnan

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking

Lefteris Loukas, Ilias Stogiannidis, Odysseas Diamantopoulos, Prodromos Malakasiotis, Stavros Vassos

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Long paper accepted to ACM ICAIF-23

LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering

结合领域专家心智模型的大语言模型增强：通过因果提示工程减少大语言模型幻觉

Boris Kovalerchuk, Brent D. Fegley

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；分类 cs.AI

AI总结 本文提出结合专家心智模型（EMM）的因果提示工程框架，通过形式化相关前置流程构建EMM，减少LLM幻觉，在三类任务中验证了方法有效性。

Comments 42 pages,4 figures, 4 tables

Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?

条款式的第三类接触：大型语言模型能否替代语言教师？

Kristina Šekrst, Ana Kovačić

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract_cn)；分类 cs.CL

AI总结 该研究评估了LLM替代语言教师提供教学反馈与解释的能力，发现其纠错能力出色但教学解释不足，AI辅助语言学习热情或超出对其教学能力的认知。

Journal ref Oxford Intersections: AI in Society (Oxford, online edn, Oxford Academic, 20 Mar. 2025 - )

FDABench: A Benchmark for Data Agents on Analytical Queries over Heterogeneous Data

FDABench：异构数据上分析查询的数据代理基准

Ziting Wang, Shize Zhang, Haitao Yuan, Jinwei Zhu, Wei Dong, Gao Cong

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；分类 cs.DB

AI总结 提出FDABench基准，通过2007个任务覆盖六种数据模态，并设计PUDDING框架自动构建数据集，以评估数据代理在异构数据上的推理能力。

Comments Accepted to KDD'26

MemoryLake on MemoryArena: A Matched Study of Agent Memory Backends

基于MemoryArena的MemoryLake研究：智能体记忆后端的匹配对比

Chaoqun Zhan, Qiang Zhou, Guannan Li, Zhenqiang Huang, Qianjin Wang

机构 * MemoryLake Team（MemoryLake团队） 纠错

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；分类 cs.AI

AI总结 该研究在MemoryArena的五个任务领域中，对比MemoryLake等三种智能体记忆后端，发现MemoryLake在数学等三个领域成功率最高，整体平均成功率领先，且性能与工作负载相关。

Comments 16 pages, 7 tables

Reconcile Once, Write Anytime: A Trust-Tiered Librarian and a Multi-Agent Writer for Drift-Free, Point-in-Time Research

一次对账，随时撰写：用于无漂移、时点研究的信任分层图书馆与多智能体撰写器

Xing Zhang, Yanwei Cui, Guanghui Wang, Peiyang He

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；分类 cs.CL

AI总结 该研究提出双层智能体系统，通过信任分层图书馆与多智能体撰写器分离知识库与报告撰写，消除报告矛盾与漂移，实验验证其在多指标、多场景下的有效性与效率优势。

Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agents

多层架构中的防御效果：对持久内存攻击状态机LLM代理的机制性评估

Jun Wen Leong

专题命中 RAG评测 ：RAG(abstract,abstract_cn)；分类 cs.AI

AI总结 本文评估了六种防御措施在九个开源模型上的延迟触发攻击效果，发现输入级和检索级过滤器效果不佳，而内存层工具门控（Memory Sandbox）显著降低攻击成功率，揭示了不同防御类别的失效原因。

Comments v4: Added double dissociation (reasoning-mode ablation), content-layer defense (RATG), loaded-corpus frontier evaluation (21 models, 3 providers, N=40), 7B judge capability bound, reproducibility validity criterion, ethics/disclosure statement. Gemini 3.1 Pro Preview 95% ASR; GPT-5.1 regression (22.5%); tripartite vendor divergence. Code: github.com/junwenleong/stateful-agent-security-eval