语言大模型 / LLM

Large Language Model Agent Personality and Response Appropriateness: Evaluation by Human Linguistic Experts, LLM-as-Judge, and Natural Language Processing Model

Eswari Jayakumar, Niladri Sekhar Dash, Debasmita Mukherjee

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Are Prompts All You Need? Evaluating Prompt-Based Large Language Models (LLM)s for Software Requirements Classification

Manal Binkhonain, Reem Alfayaz

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Comments 33 pages, 12 figures

Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy

Daniil Filienko, Yinzhou Wang, Caroline El Jazmi, Serena Xie, Trevor Cohen, Martine De Cock, Weichao Yuwen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for AMIA 2024 proceedings

Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboard

Oguzhan Topsakal, Colby Jacob Edell, Jackson Bailey Harper

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data

Yubin Kim, Xuhai Xu, Daniel McDuff, Cynthia Breazeal, Hae Won Park

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(title)；prompting(abstract)

FedEval-LLM: Federated Evaluation of Large Language Models on Downstream Tasks with Collective Wisdom

Yuanqin He, Yan Kang, Lixin Fan, Qiang Yang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments In Progress

MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models

Yilin Wen, Zifeng Wang, Jimeng Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 8 figures, 12 tables

An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide

Oluwole Fagbohun, Rachel M. Harrison, Anton Dereventsov

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL、cs.AI、cs.LG

LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models

Minsuk Kahng, Ian Tenney, Mahima Pushkarna, Michael Xieyang Liu, James Wexler, Emily Reif, Krystal Kallarackal, Minsuk Chang, Michael Terry, Lucas Dixon

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Large Language Model (LLM) Bias Index -- LLMBI

Abiodun Finbarrs Oketunji, Muhammad Anas, Deepthi Saina

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Exploring Prompting Large Language Models as Explainable Metrics

Ghazaleh Mahmoudi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, Eval4NLP 2023

Journal ref Proceedings of the 4th Workshop on Evaluation and Comparison for NLP systems (2023)

Prompting Vision Language Model with Knowledge from Large Language Model for Knowledge-Based VQA

Yang Zhou, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；LLM(abstract)

SPLIT: Cross-Lingual Empathy and Cultural Grounding in English and Ukrainian LLM Responses

SPLIT：英语和乌克兰语LLM回应中的跨语言共情与文化根基

Anna Chorna

专题命中 评测与基准 ：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 提出SPLIT基准测试，评估LLM在英语和乌克兰语危机情境下的共情准确性、语言自然性和文化根基，发现模型在乌克兰语中表现下降，且人类与AI评估者一致性弱。

Comments 19 pages, 5 figures, 3 tables. Benchmark paper introducing SPLIT for evaluating empathy, linguistic naturalness, and cultural grounding in English and Ukrainian LLM responses

Spark-LLM-Eval: A Distributed Framework for Statistically Rigorous Large Language Model Evaluation

Spark-LLM-Eval: 一个用于统计严谨大语言模型评估的分布式框架

Subhadip Mitra

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

AI总结 本文提出Spark-LLM-Eval，一个基于Apache Spark的分布式评估框架，通过数据并行处理实现大规模LLM评估，强调统计严谨性，提供置信区间和显著性检验，并利用Delta Lake实现响应缓存以降低评估成本。

Comments 16 pages, 2 figures, 6 tables. Open source: https://github.com/bassrehab/spark-llm-eval. Cross-list requested: cs.CL, cs.LG

ECG-LLM -- training and evaluation of domain-specific large language models for electrocardiography

Lara Ahrens, Wilhelm Haverkamp, Nils Strodthoff

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 34 pages, 8 figures, code available at https://github.com/AI4HealthUOL/ecg-llm

Air Traffic Control Using Large Language Models: Prompt Engineering, Architecture, and Evaluation

使用大语言模型的空中交通管制：提示工程、架构与评估

Mahyar Ghazanfari, Matthias Casanova, Jordan Kam, Alex Zongo, Peng Wei, Torsten Darrell, Alexandre Bayen

机构 * The George Washington University（乔治华盛顿大学） ； California Institute of Technology（加州理工学院） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 该研究针对ATC仍以人工为主的问题，设计5种提示结构并在9种LLMs上实验，发现简洁提示表现最佳，注入正确历史可修复脚本严格提示的错误，明确了LLM辅助ATC的路径与局限。

Comments 39 pages, 12 figures, 7 tables

Communicating Credit Risk with Large Language Models: Evaluation of Explanations from Standard and Alternative Data-Based Models

利用大语言模型传达信用风险：基于标准数据与替代数据模型的解释评估

Sahab Zandi, Noah Kostesku, Christophe Mues, María Óskarsdóttir, Cristián Bravo

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 本研究探究大语言模型能否作为解释层转化信用风险模型的事后解释，构建三类流程并采用三类LLM，发现证据表征是解释质量的关键制约，专业人士对证据标准要求更严格。

Comments 49 pages, 14 figures

Effects of Answer Format Variation on Gender Bias in Large Language Models

答案格式变化对大型语言模型中性别偏见的影响

Ksenia Merzlyakova, Sebastian Padó, Franziska Weeber

机构 * Institute for Natural Language Processing, University of Stuttgart（斯图加特大学自然语言处理研究所） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AI总结 该研究探讨答案格式变化对LLMs性别偏见测量的影响，通过评估三种指令微调模型在不同格式下的表现，发现格式会显著改变测量结果，强调需将答案格式纳入LLM评估。

Comments 6th Workshop on Computational Linguistics for the Political and Social Sciences (CPSS 2026)

CACSurv: Concordance-Aligned Comparative Learning with Large Language Models for Cancer Survival Prediction

CACSurv：基于大型语言模型的一致性对齐比较学习用于癌症生存预测

Tianqi Xiang, Qixiang Zhang, Xinpeng Ding, Yi Li, Xiaomeng Li

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 该研究针对癌症生存预测中LLM时间回归的公式与监督不匹配问题，提出CACSurv框架，构建TCGA-SurvReport基准，在6个TCGA癌症队列上的平均C指数达0.722，显著优于现有模型与基线。

CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models

CORDA：面向大语言模型的以伤害为核心的分层道德推理基准

Siddarth Singh, Victoria Williams, Simon Rosen, Ebenezer Gelo, Helen Sarah Robertson, Ibrahim Suder, Benjamin Rosman, Geraud Nangue Tasse, Steven James

机构 * University of the Witwatersrand（威特沃特斯兰德大学） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 研究提出CORDA基准，评估大语言模型的以伤害为核心的分层道德推理，发现多数模型优先避免直接个人伤害，部分模型无法遵循指定优先级，该基准填补了LLM道德评估的核心空白。

Comments 8 pages, 6 figures. Accepted at the Fourth International Workshop on Value Engineering in AI (VALE 2026), affiliated with IJCAI-ECAI 2026

Verication-driven closed-loop multi-agent large language modelframework for code-compliant structural design

验证驱动的闭环多智能体大语言模型框架：面向符合代码规范的结构设计

Jianbin Luo, Weibin Lin, Yiran Lin, Qing Wei, Wei Guo

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 本研究提出验证驱动的闭环多智能体LLM框架，通过耦合有限元验证系统与双节点结构提升结构设计的代码合规性，开源基准与脚本，性能提升显著且具可复现性。

Comments 20 pages, 21 figures

An Agentic AI Framework Overcomes Fundamental Limitations of Large Language Models for Glaucoma Detection from Fundus Photography

一种智能体AI框架克服了大型语言模型在眼底照相青光眼检测中的基本局限

Jalil Jalili, Hossein Taghizad, Anuwat Jiravarnsirikul, Christopher Bowd, Akram Belghith, Raheleh Kafieh, Christopher A. Girkin, Sally L. Baxter, Robert N. Weinreb, Linda M. Zangwill, Mark Christopher

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 该研究开发的智能体AI框架整合LLM与专用深度学习工具，提升了眼底照相青光眼检测的准确率、一致性，纠正了仅用LLM的缺陷，具有通用性，或推动医学AI向多智能体系统转变。

AI Playing Business Games: Benchmarking Large Language Models on Managerial Decision-Making in Dynamic Simulations

AI 玩商业游戏：在动态模拟中对大型语言模型的管理决策能力进行基准测试

Berdymyrat Ovezmyradov

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 本研究提出一种用于LLM基准测试的可复现开放获取管理模拟器，评估Gemini等五个LLMs在动态商业模拟中的多步骤战略决策能力，为长期决策评估提供新途径。

Comments 34 pages, 7 figures, 3 tables

Large language models improve physician accuracy but lead to false reliance

大型语言模型可提高医师准确率，但会导致错误依赖

Tirtha Chanda, Christoph Wies, Franziska Schramm, Carina Nogueira Garcia, Nicolas B. Merl, Martin J. Hetz, Jochen S. Utikal, Phillip Tschandl, Cristian Navarrete-Dechent, Alexander Thiem, Jakob N. Kather, Consortium, Titus J. Brinker

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 本研究开发智能体式检索增强型LLM CORA，发现其可将46名医师的诊断准确率从70.8%提升至82.6%，但存在错误答案获引用支持时医师抵抗大幅下降的安全风险。

Some hypotheses on how chatbots work in problem-solving-driven conversations. Large Language Models as confirmation of the Innovation Illusion

关于聊天机器人在问题解决驱动对话中如何工作的一些假设：大型语言模型作为创新幻觉的确认

S. F. M. van Vlijmen, H. D. Lethe

机构 * S.F.M. van Vlijmen and H.D. Lethe jr（S.F.M. van Vlijmen 和 H.D. Lethe jr） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 本文提出聊天机器人作为对话伙伴的本质，基于聚合动力学、认知语言学等理论，假设LLM训练数据仅部分模仿人类思维，并得出结论：基础聊天机器人无法成为与人类匹敌的思考伙伴。

Comments Changes made over the versions do NOT concern the argument or conclusion. The changes do concern: typo's, better phrasing, extra references, making the abstract fit the length demands without losing the main points to be made. 42 pages, 3 figures, submitted to Transmathematica

Human Grounded Evaluation of Large Language Models for Optical Network Automation

用于光网络自动化的大语言模型的人工基础评估

Kiarash Rezaei, Omran Ayoub, Paolo Monti, Carlos Natalino

机构 * University of Applied Sciences and Arts of Southern Switzerland（瑞士南瑞士应用科学与艺术大学） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 研究针对光网络自动化中LLMs输出质量和成本差异大的问题，提出HuGLEN评估管道，结合LLM评判器和专家评级，用QES排名，用于转换XAI模型输出，结果显示中等规模LLM的QES最高，减轻人工标注负担，助力模型选择。

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

SycoEval-EM：急诊护理模拟临床场景中大语言模型的谄媚评估

Dongshen Peng, Yi Wang, Austin Schoeffler, Sun-ha Hong, Brian Suffoletto, David Kim, Carl Preiksaitis, Christian Rose

机构 * UNC Chapel Hill（UNC夏洛特山分校） ； University of Waterloo（滑铁卢大学） ； Stanford University（斯坦福大学） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 提出SycoEval-EM多智能体模拟框架，评估19个LLM在急诊医学中对患者说服的鲁棒性，发现谄媚率呈双峰分布，静态医学基准无法预测安全表现。

Comments 19 pages, 10 figures

OntoLearner: A Modular Python Library for Ontology Learning with Large Language Models

OntoLearner: 一个用于大语言模型本体学习的模块化Python库

Hamed Babaei Giglou, Jennifer D'Souza, Andrei Aioanei, Nandana Mihindukulasooriya, Sören Auer

机构 * TIB – Leibniz Information Centre for Science and Technology（TIB – 莱布尼茨科学与技术信息中心） ； L3S Research Center, Leibniz University of Hannover（L3S研究中心，莱布尼茨汉诺威大学） ； IBM Research（IBM研究院） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 提出OntoLearner框架，统一本体访问、LLM驱动学习管道和标准化基准，通过跨领域大规模实验揭示本体学习的主要瓶颈是模型知识编码与本体组织的结构性不匹配。

Comments 30 pages. Under review at Nature Communications. This version is reformatted with a different section structure; content is unchanged

StatEval: A Comprehensive Benchmark for Large Language Models in Statistics

StatEval：大型语言模型在统计学中的综合基准

Yuchen Lu, Run Yang, Yichen Zhang, Shuguang Yu, Ziwei Wang, Jiayi Xiang, Wenxin E, Changyu Zhu, Fan Zhou

机构 * Shanghai University of Finance and Economics（上海财经大学） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AI总结 提出StatEval基准，包含10万+问题，覆盖本科到研究级统计推理，并开发TRACE管道和自适应评分方法，揭示LLM在基础任务上表现尚可但研究级推理薄弱。

BrainAgent: A Large Language Model-Driven Multi-Agent Framework for Autonomous Brain Signal Understanding

BrainAgent：一种大语言模型驱动的多智能体框架，用于自主脑信号理解

Yangxuan Zhou, Sha Zhao, Jiquan Wang, Shijian Li, Gang Pan

专题命中 评测与基准 ：LLM(summary_cn,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

AI总结 提出LLM驱动的多智能体框架BrainAgent，通过分层架构将自然语言意图转化为可执行的处理流水线，实现脑信号分析的自动化与民主化。

Comments 22 pages, 11 figures