语言大模型 / LLM - arXivDaily 专题

2305.11116 2023-05-19 cs.CV cs.CL 88%

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, William Yang Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2305.03429 2023-05-17 cs.CL 88%

Simulating H.P. Lovecraft horror literature with the ChatGPT large language model

Eduardo C. Garrido-Merchán, José Luis Arroyo-Barrigüete, Roberto Gozalo-Brizuela

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2304.12749 2023-05-02 cs.CR cs.LG 88%

Blockchain Large Language Models

Yu Gai, Liyi Zhou, Kaihua Qin, Dawn Song, Arthur Gervais

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

2304.09957 2023-04-21 cs.CL 88%

Low-resource Bilingual Dialect Lexicon Induction with Large Language Models

Ekaterina Artemova, Barbara Plank

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to NoDaLiDa 2023

2303.07304 2023-03-14 cs.CL 88%

Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Research

Nigel Williams, Stanislav Ivanov, Dimitrios Buhalis

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 14 pages

2301.03797 2023-02-10 cs.SE cs.LG 88%

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Toufique Ahmed, Supriyo Ghosh, Chetan Bansal, Thomas Zimmermann, Xuchao Zhang, Saravan Rajmohan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments Accepted at International Conference on Software Engineering (ICSE-2023)

2112.11941 2022-10-06 cs.CL 88%

CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

Jörg Frohberg, Frank Binder

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 10 pages including references, plus 5 pages appendix. Edits for version 3 vs LREC 2022: Point out human baseline in abstract (also to match arxiv abstract), fix affiliation apergo.ai, and fix a recurring typo

Journal ref Proceedings of the 13th Language Resources and Evaluation Conference (LREC 2022), Marseille, France pp. 2126-2140 (2022) https://aclanthology.org/2022.lrec-1.229/

2206.11484 2022-07-11 cs.CL cs.CY 88%

Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models

Virginia K. Felkner, Ho-Chun Herbert Chang, Eugene Jang, Jonathan May

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to Queer in AI Workshop @ NAACL 2022. Updated 07/07 with minor typographical fixes

2202.13169 2022-05-05 cs.PL cs.CL 88%

A Systematic Evaluation of Large Language Models of Code

Frank F. Xu, Uri Alon, Graham Neubig, Vincent J. Hellendoorn

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments DL4C@ICLR 2022, and MAPS@PLDI 2022

2204.09391 2022-04-21 cs.CL 88%

You Are What You Write: Preserving Privacy in the Era of Large Language Models

Richard Plant, Valerio Giuffrida, Dimitra Gkatzia

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2108.07732 2021-08-18 cs.PL cs.LG 88%

Program Synthesis with Large Language Models

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, Charles Sutton

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments Jacob and Augustus contributed equally

2312.10059 2023-12-19 cs.CY 88%

A collection of principles for guiding and evaluating large language models

Konstantin Hebenstreit, Robert Praas, Matthias Samwald

专题命中评测与基准：language model(title,abstract)；large language model(title,abstract)

Comments Accepted at Socially Responsible Language Modelling Research (SoLaR) workshop, NeurIPS 2023 (https://openreview.net/forum?id=8iXdNXW34d). Based on previous manuscript version: doi:10.2139/ssrn.4446991

2608.22793 2026-08-25 cs.CL cs.AI 新提交 88%

TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents

TRACE：面向一致性、极限感知的自进化技能库

Wenhao Wu, Menghao Zhang, Xin Wang, Zhi Wang, Kun Shao, Jian Luan

机构 * Xiaomi Inc.（小米公司）； Nanjing University（南京大学）； Beijing University of Posts and Telecommunications（北京邮电大学）； Tsinghua University（清华大学）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI

AI总结 TRACE是一种无需修改模型权重的自进化技能库方法，通过轨迹对比进化优化技能，在CAR-bench任务上显著提升LLM智能体的一致性性能，缩小潜在与可靠性能的差距。

Comments 9 pages, 5 figures, 2 tables

2608.21544 2026-08-25 cs.CL cs.AI 新提交 88%

Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents

权重中遗忘，工具中恢复：面向大语言模型智能体的智能体工具遗忘

Baicheng Chen, Zheyuan Liu, Jingyu Zhang, Kaize Ding, Ningshan Ma, Yue Huang, Meng Jiang

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； University of Notre Dame（圣母大学）； Johns Hopkins University（约翰斯·霍普金斯大学）； Northwestern University（西北大学）； MIT（麻省理工学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结该研究针对大语言模型智能体的工具介导恢复问题，提出两阶段的Agentic Tool Unlearning框架，在RWKU和MUSE数据集上实现了更好的遗忘与保留效用平衡。

2608.21365 2026-08-25 cs.CL cs.AI cs.IR 新提交 88%

KSE-Web: An Analysis of Hybrid Retrieval and LLM-Assisted Query Expansion for Low-Resource Khmer Semantic Search

KSE-Web：面向低资源高棉语语义搜索的混合检索与大语言模型辅助查询扩展分析

Nimol Thuon

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI

AI总结本文针对低资源高棉语语义搜索，构建专属数据集，评估多种检索方法，发现BM25性能最优，LLM辅助查询扩展未提升效果但模型规模影响显著，为相关研究提供基础。

2608.10008 2026-08-25 cs.IR cs.CL cs.LG 版本更新 88%

Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness

大语言模型推荐系统是否知道自己在产生幻觉？针对目录保真度的置信度校准审计

Srijith Ravikumar

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.LG

AI总结该研究审计了四个零样本LLM推荐系统的幻觉率与置信度校准，发现其系统性置信不足，提出conformal弃权阈值效果有限，建议审计需同时报告校准与OOD率并采用锚定目录的提示。

Comments Accepted at CIKM 2026 (short paper). v2 corrects the catalog-membership matcher, which scored truncated catalog entries as perfect matches to longer hallucinated titles. One finding reverses: v1 reported all four models under-confident in all twelve cells; v2 finds the catalog sets the direction. Amazon Toys hallucination moves from 4.5-8.3% to 49.3-61.0%. Cite v2 or later; v1 is superseded

2608.20376 2026-08-24 cs.CL cs.LG 新提交 88%

TH-GNN: Heterogeneous Temporal Graph Neural Networks for LLM-Agent Shilling Attack Detection

TH-GNN：用于检测大语言模型智能体刷单攻击的异质性时序图神经网络

Shivam Swarup, Divya Prakash Shrivastava, Rakesh Thakur

机构 * JAIN (Deemed to be University)（JAIN（被认定大学））； Zayed University（扎耶德大学）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.LG

AI总结本文提出TH-GNN，一种异质性时序图神经网络，联合建模时序、结构与语义信号，在5类攻击族和4个基准数据集上检测LLM智能体刷单攻击，总体平均F1值达0.870，性能优于纯文本基线模型。

2608.18091 2026-08-20 cs.CL cs.AI 新提交 88%

Self- and Other-Labels Induce Bidirectional Bias in LLM Judges

自标签与他标签诱导大语言模型评判器产生双向偏差

Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, Seohyon Jung

机构 * School of Digital Humanities and Computational Social Sciences, KAIST（韩国科学技术院数字人文与计算社会科学学院）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI

AI总结该研究针对LLM评判器的自偏好混杂问题，通过评估带模型特征的叙事约束选择，发现控制质量后自偏好消失，而自/他标签会双向影响评分，明确了作者归属是评估偏差的独立驱动因素。

2602.13452 2026-08-13 cs.CL cs.AI 版本更新 88%

LLM-Powered Automatic Translation and Urgency in Crisis Scenarios

基于大型语言模型的自动翻译与危机场景中的紧急性

Belu Ticona, Antonis Anastasopoulos

机构 * George Mason University（乔治·马歇尔大学）； George Mason University – Archimedes AI Research Unit（乔治·马歇尔大学阿基米德人工智能研究单元）； Athena RC, Greece（雅典RC，希腊）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本文研究了大型语言模型在危机场景中翻译的性能，发现其在保持紧急性方面存在显著缺陷，强调了对危机通信技术的特殊评估需求。

Comments Accepted to ISCRAM 2026

2604.16706 2026-08-12 cs.AI cs.CL cs.MA 版本更新 88%

Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents

评估工具使用语言代理：裁判可靠性、错误传播和运行时缓解在AgentProp-Bench中

Bhaskar Gurram

机构 * Zasti Inc.（Zasti公司）

专题命中评测与基准：language agent(title)；LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)

AI总结本文通过AgentProp-Bench评估工具使用语言代理的可靠性，发现基于子字符串的裁判与人类标注一致性较低，但三模型集成能提高一致性，同时发现参数注入导致错误传播的概率较高，运行时拦截器在GPT-4o-mini上有效减少幻觉，但对Gemini-2.0-Flash无显著影响。

Comments 11 pages, 4 figures, 8 tables. Code and data: https://github.com/bhaskargurram-ai/agenthallu-bench

2607.01236 2026-08-11 cs.CL cs.AI 版本更新 88%

Safeguarding LLM Agents from Misalignment through Provenance Analysis

通过溯源分析保护LLM智能体免受失调影响

Yining She, Yiliang Liang, Eunsuk Kang

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中评测与基准：LLM(title,title_cn)；分类 cs.CL、cs.AI

AI总结提出基于溯源分析的ProvenanceGuard框架，通过多阶段流水线检测工具调用前的三种失调类型，在Agent-SafetyBench和WorkBench上显著降低失调错误率并减少不必要的干预。

2605.26937 2026-08-11 cs.CL cs.AI 版本更新 88%

Beyond Questions: Evaluating LLM's Knowledge Expression

超越问题：评估大型语言模型（实际）知道什么

Luca Giordano, Simon Razniewski

机构 * ScaDS.AI Dresden/Leipzig & TU Dresden, Germany（ScaDS.AI 德尔布兰德/莱比锡及德累斯顿技术大学，德国）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结提出开放知识评估新范式，通过开放式提示（如“告诉我关于M.L. King的一切”）评估模型自然表达的知识，并构建BeQu基准测试10,000个实体。

2605.12530 2026-08-11 cs.CL cs.AI cs.CY 版本更新 88%

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

针对LLM公平性的在 situ 行为评估，而非标准化测试分数

Zeyu Tang, Sang T. Truong, Deonna Owens, Shreyas Sharma, Yibo Jacky Zhang, Brando Miranda, Sanmi Koyejo

专题命中评测与基准：LLM(title,title_cn)；分类 cs.CL、cs.AI

AI总结本文提出MAC-Fairness框架，通过多轮对话评估模型在不同身份下的行为变化，揭示稳定的行为特征，超越传统标准化测试的局限。

Comments Accepted to COLM 2026

2608.05741 2026-08-07 cs.CL cs.AI 新提交 88%

Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration

一次响应，始终是响应：通过潜在提示恢复检测大语言模型生成的文本

Hongrui Bao, Yubing Ren, Yanan Cao, Jinhan You, Fang Fang, Shi Wang

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）； College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）； Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）

专题命中评测与基准：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结针对现有零样本LLM生成文本检测器未考虑LLM训练过程的问题，提出无训练检测器EchoPrompt，通过潜在提示恢复实现零样本检测，在零样本检测器中达SOTA性能且稳健性强。

Comments 17 pages, 7 figures

2608.05155 2026-08-07 cs.CL cs.AI 新提交 88%

Beyond Sentiment: Comparing Traditional NLP and LLM-Based Multi-Dimensional Analysis for Political News Evaluation

超越情感：对比传统NLP与基于大语言模型的多维分析在政治新闻评估中的应用

Maryam Fooladi, Federico Bottino

机构 * Kakashi Ventures Accelerator (KVA)（卡卡西风险投资加速器（KVA））； Newjee（新吉）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI

AI总结本文对比传统NLP的RoBERTa情感分析与基于LLM的多维框架分析，发现前者存在“中性坍缩”局限，后者可捕捉政治话语多维特征，更适配SSH研究需求。

Comments Accepted at PoliticalNLP 2026, the 3rd Workshop on Natural Language Processing for Political Sciences, co-located with LREC 2026. 10 pages, 3 figures

2509.19358 2026-08-06 cs.CL cs.AI 88%

Benchmarking and Improving LLM Robustness for Personalized Generation

Chimaobi Okite, Naihao Deng, Kiran Bodipati, Huaidian Hou, Joyce Chai, Rada Mihalcea

机构 * University of Michigan（密歇根大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments First draft. First camera-ready version

Journal ref Findings of the Association for Computational Linguistics: EMNLP 2025, 2025

2608.03219 2026-08-05 cs.AI cs.CL 新提交 88%

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

可达性并非可实现性：追溯大语言模型基准测试增益的来源

Yanchao Li, Wanhao Liu, Jiaqing Xie, Ben Gao, Yanbo Wang, Tianfan Fu, Yuqiang Li

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI

AI总结该研究针对LLM基准测试增益，建立问题级审计方法，发现可实现性与可达性常不同步，提出能力扩展声明需报告匹配条件下的可实现性能与可达性。

2508.16949 2026-08-04 cs.LG cs.AI 88%

Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning

突破探索瓶颈：面向通用大语言模型推理的Rubric引导强化学习

Yang Zhou, Sunzhu Li, Shunyu Liu, Wenkai Fang, Kongcheng Zhang, Jiale Zhao, Jingwen Yang, Yihe Zhou, Jianwei Lv, Tongya Zheng, Hengtong Lu, Wei Chen, Yan Xie, Mingli Song

机构 * Zhejiang University（浙江大学）； Li Auto Inc.（力汽车公司）； Nanyang Technological University（南洋理工大学）； The Chinese University of Hong Kong（香港中文大学）； Hangzhou City University（杭州市大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结本文提出Rubric-Scaffolded Reinforcement Learning，通过引入检查表式评分标准作为探索和利用的指导框架，突破通用大语言模型推理的探索瓶颈，提升推理能力。

2607.26397 2026-07-30 cs.CL cs.LG q-bio.QM 新提交 88%

Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification

推理前的知识：EC-Reason-Bench，一种用于大语言模型酶分类的无训练诊断基准

Linyu Li, Zhi Jin, Yichi Zhang, Dongming Jin, Yuanpeng He, Huanyao Zhang, Xuan Zhang, Gadeng Luosang, Nyima Tashi

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.LG

AI总结该研究针对通用LLM酶分类中EC编号二至四级准确率近乎为零的问题，提出无训练诊断基准EC-Reason-Bench，分解四个维度评估，发现外部知识优先于推理等关键结论。

2607.01240 2026-07-30 cs.CL cs.AI 版本更新 88%

Prompt Framing Distorts Count Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoring

提示框架扭曲了基于计数的LLM错误检测评估：来自数字锚定的证据

Dekun Yang

机构 * Zhejiang University（浙江大学）

专题命中评测与基准：LLM(title,title_cn)；分类 cs.CL、cs.AI

AI总结本文发现基于计数的F1分数可能因提示中的数字锚定而虚高，与跨度定位能力脱节，提出ErrorBench压力测试协议，并建议评估时应避免预置错误计数并报告跨度感知指标。

Comments 15 pages, 6 figures, 12 tables. Preprint under review

1. 评测与基准 32208 篇

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Simulating H.P. Lovecraft horror literature with the ChatGPT large language model

Blockchain Large Language Models

Low-resource Bilingual Dialect Lexicon Induction with Large Language Models

Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Research

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models

A Systematic Evaluation of Large Language Models of Code

You Are What You Write: Preserving Privacy in the Era of Large Language Models

Program Synthesis with Large Language Models

A collection of principles for guiding and evaluating large language models

TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents

Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents

KSE-Web: An Analysis of Hybrid Retrieval and LLM-Assisted Query Expansion for Low-Resource Khmer Semantic Search

Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness

TH-GNN: Heterogeneous Temporal Graph Neural Networks for LLM-Agent Shilling Attack Detection

Self- and Other-Labels Induce Bidirectional Bias in LLM Judges

LLM-Powered Automatic Translation and Urgency in Crisis Scenarios

Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents

Safeguarding LLM Agents from Misalignment through Provenance Analysis

Beyond Questions: Evaluating LLM's Knowledge Expression

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration

Beyond Sentiment: Comparing Traditional NLP and LLM-Based Multi-Dimensional Analysis for Political News Evaluation

Benchmarking and Improving LLM Robustness for Personalized Generation

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning

Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification

Prompt Framing Distorts Count Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoring