语言大模型 / LLM - arXivDaily 专题

2510.19032 2025-10-23 cs.CL cs.CY cs.HC 85%

When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation

Abeer Badawi, Elahe Rahimi, Md Tahmid Rahman Laskar, Sheri Grach, Lindsay Bertrand, Lames Danok, Jimmy Huang, Frank Rudzicz, Elham Dolatabadi

机构 * York University（约克大学）； Vector Institute（向量研究所）； Dalhousie University（达尔豪斯大学）； IWK Health Hospital（IWK健康医院）； King’s College London（伦敦国王学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2510.19025 2025-10-23 cs.DB cs.AI 85%

FlexiDataGen: An Adaptive LLM Framework for Dynamic Semantic Dataset Generation in Sensitive Domains

Hamed Jelodar, Samita Bai, Roozbeh Razavi-Far, Ali A. Ghorbani

机构 * Canadian Institute for Cybersecurity（加拿大网络安全研究所）； Faculty of Computer Science（计算机科学学院）； University of New Brunswick（新不伦瑞克大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2506.11791 2025-10-23 cs.LG cs.CR 85%

SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks

Hwiwon Lee, Ziqi Zhang, Hanxiao Lu, Lingming Zhang

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2510.03417 2025-10-22 cs.CR cs.AI 85%

NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks

Javad Rafiei Asl, Sidhant Narula, Mohammad Ghasemigol, Eduardo Blanco, Daniel Takabi

机构 * Old Dominion University（旧 Dominion 大学）； University of Arizona（亚利桑那大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments This paper has been accepted in the main conference proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025). Javad Rafiei Asl and Sidhant Narula are co-first authors

2506.03655 2025-10-22 cs.CY cs.CL 85%

Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability

Lorraine Saju, Arnim Bleier, Jana Lasser, Claudia Wagner

机构 * GESIS – Leibniz Institute for the Social Sciences（GESIS社会科学院研究所）； RWTH Aachen University（亚琛RWTH大学）； University of Graz（格拉茨大学）； Complexity Science Hub（复杂性科学中心）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2510.17698 2025-10-21 cs.CL 85%

Towards Mining Effective Pedagogical Strategies from Learner-LLM Educational Dialogues

Liqun He, Manolis Mavrikis, Mutlu Cukurova

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2510.17491 2025-10-21 cs.CL 85%

Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

Yihong Tang, Kehai Chen, Liang Yue, Jinxin Fan, Caishen Zhou, Xiaoguang Li, Yuyang Zhang, Mingming Zhao, Shixiong Kai, Kaiyang Guo, Xingshan Zeng, Wenjing Cun, Lifeng Shang, Min Zhang

机构 * School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China（计算机科学与技术学院，哈尔滨工业大学，深圳，中国）； Huawei Technologies Co., Ltd.（华为技术有限公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2411.10213 2025-10-21 cs.SE cs.AI 85%

An Empirical Study on LLM-based Agents for Automated Bug Fixing

Xiangxin Meng, Zexiong Ma, Pengfei Gao, Chao Peng

机构 * ByteDance（字节跳动）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.16081 2025-10-21 cs.CY cs.AI 85%

SARHAchat: An LLM-Based Chatbot for Sexual and Reproductive Health Counseling

Jiaye Yang, Xinyu Zhao, Tianlong Chen, Kandyce Brennan

机构 * University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 5 pages, 1 figure

2510.15081 2025-10-20 cs.CL cs.SI 85%

A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling

Shiyu Ji, Farnoosh Hashemi, Joice Chen, Juanwen Pan, Weicheng Ma, Hefan Zhang, Sophia Pan, Ming Cheng, Shubham Mohole, Saeed Hassanpour, Soroush Vosoughi, Michael Macy

机构 * Cornell University（康奈尔大学）； Georgia Institute of Technology（佐治亚理工学院）； Dartmouth College（达特茅斯学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments The first two authors contributed equally

2510.14966 2025-10-17 cs.LG stat.ML 85%

Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores

Zachary Robertson

机构 * Stanford University（斯坦福大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 9 pages, 2 figures

2510.14756 2025-10-17 cs.CL 85%

Pluto: A Benchmark for Evaluating Efficiency of LLM-generated Hardware Code

Manar Abdelatty, Maryam Nouh, Jacob K. Rosenstein, Sherief Reda

机构 * Department of Electrical and Computer Engineering（电气与计算机工程系）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2509.26072 2025-10-15 cs.CL 85%

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge

Arash Marioriyad, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 9 Pages, 5 Tables, 1 Figures

Journal ref 39th Conference on Neural Information Processing System, 2025, Workshop: Reliable ML from Unreliable Data

2510.11434 2025-10-14 cs.CL 85%

Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content

Dana Sotto Porat, Ella Rabinovich

机构 * The Academic College of Tel Aviv--Yaffo（特拉维夫-亚法夫学术学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ECAI2025 (Identity-Aware AI workshop)

2502.01860 2025-10-13 cs.SE cs.LG 85%

SWE-Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering

Zhimin Zhao

机构 * School of Computing Queen's University Kingston, Canada（计算学院皇后大学加拿大金斯顿）

专题命中评测与基准：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Check the online arena at https://huggingface.co/spaces/SWE-Arena/Software-Engineering-Arena

2502.08638 2025-10-10 cs.CL 85%

Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples

Andrianos Michail, Simon Clematide, Rico Sennrich

机构 * University of Zurich（苏黎世大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments To appear in EMNLP2025 Findings

2503.01493 2025-10-10 cs.CL 85%

Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting

Fajri Koto, Rituraj Joshi, Nurdaulet Mukhituly, Yuxia Wang, Zhuohan Xie, Rahul Pal, Daniil Orel, Parvez Mullah, Diana Turmakhan, Maiya Goloburda, Mohammed Kamran, Samujjwal Ghosh, Bokang Jia, Jonibek Mansurov, Mukhammed Togmanov, Debopriyo Banerjee, Nurkhan Laiyk, Akhmed Sakip, Xudong Han, Ekaterina Kochmar, Alham Fikri Aji, Aaryamonvikram Singh, Alok Anil Jadhav, Satheesh Katipomu, Samta Kamboj, Monojit Choudhury, Gurpreet Gosal, Gokulakrishnan Ramakrishnan, Biswajit Mishra, Sarath Chandran, Avraham Sheinin, Natalia Vassilieva, Neha Sengupta, Preslav Nakov

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）； Inception ； Cerebras Systems（Cerebras系统）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at COLM 2025

2503.00096 2025-10-10 q-bio.QM cs.AI 85%

BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology

Ludovico Mitchener, Jon M Laurent, Alex Andonian, Benjamin Tenmann, Siddharth Narayanan, Geemi P Wellawatte, Andrew White, Lorenzo Sani, Samuel G Rodriques

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 8 main text pages, 5 main figures

2510.07083 2025-10-09 cs.CL 85%

All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generations

Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Benjamin Van Durme, Nick Craswell

机构 * Johns Hopkins University（约翰霍普金斯大学）； Microsoft（微软）； University of Strathclyde（斯特拉思克莱德大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2508.14146 2025-10-09 cs.CL 85%

MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automation

Xian Gao, Jiacheng Ruan, Zongyun Zhang, Jingsheng Gao, Ting Liu, Yuzhuo Fu

机构 * Shanghai Jiao Tong University（上海交通大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Work in progress

2510.06253 2025-10-09 cs.CY cs.AI 85%

LLM-Driven Rubric-Based Assessment of Algebraic Competence in Multi-Stage Block Coding Tasks with Design and Field Evaluation

Yong Oh Lee, Byeonghun Bang, Sejun Oh

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.06231 2025-10-09 cs.CV cs.CL 85%

CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generation

Mingzhe Zheng, Dingjie Song, Guanyu Zhou, Jun You, Jiahao Zhan, Xuran Ma, Xinyuan Song, Ser-Nam Lim, Qifeng Chen, Harry Yang

机构 * The Hong Kong University of Science and Technology（香港科学与技术大学）； Lehigh University（莱斯大学）； Fudan University（复旦大学）； Emory University（埃默里大学）； University of Central Florida（中央佛罗里达大学）

专题命中评测与基准：LLM(title)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 24 pages, 9 figures

2510.04764 2025-10-09 cs.CL 85%

Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models

Raha Askari, Sina Zarrieß, Özge Alacam, Judith Sieker

机构 * Department of Humanities, University of Turin（都灵大学人文学科系）； Computational Linguistics, Department of Linguistics, Bielefeld University（比勒菲尔德大学语言学系计算语言学）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted for the BabyLM workshop in EMNLP 2025

2510.06018 2025-10-08 cs.CL 85%

Evaluating The Impact of Stimulus Quality in Investigations of LLM Language Performance

Timothy Pistotti, Jason Brown, Michael Witbrock

机构 * University of Auckland（奥克兰大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Presented at https://brigap-workshop.github.io/ Information to be updated upon publication of proceedings

2510.06001 2025-10-08 cs.CL 85%

Exploring Gaps in the APS: Direct Minimal Pair Analysis in LLM Syntactic Assessments

Timothy Pistotti, Jason Brown, Michael Witbrock

机构 * University of Auckland（奥克兰大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Presented at the https://brigap-workshop.github.io/ Information to be updated after publication of proceedings

2409.15436 2025-10-08 cs.HC cs.AI 85%

Ads that Talk Back: Implications and Perceptions of Injecting Personalized Advertising into LLM Chatbots

Brian Jay Tang, Kaiwen Sun, Noah T. Curran, Florian Schaub, Kang G. Shin

机构 * University of Michigan（密歇根大学）； Indiana University Bloomington（印第安纳大学布卢明顿分校）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Journal ref Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2025, (UbiComp)

2510.05605 2025-10-08 cs.CR cs.AI 85%

AutoPentester: An LLM Agent-based Framework for Automated Pentesting

Yasod Ginige, Akila Niroshan, Sajal Jain, Suranga Seneviratne

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments IEEE TrustCom 2025 10 pages

2510.05139 2025-10-08 cs.CL 85%

NLD-LLM: A systematic framework for evaluating small language transformer models on natural language description

Hamed Jelodar, Mohammad Meymani, Parisa Hamedi, Tochukwu Emmanuel Nwankwo, Samita Bai, Roozbeh Razavi-Far, Ali A. Ghorbani

机构 * Faculty of Computer Science（计算机科学学院）； University of New Brunswick（新不伦瑞克大学）

专题命中评测与基准：LLM(title,abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

2510.03862 2025-10-07 cs.SE cs.AI 85%

Designing Empirical Studies on LLM-Based Code Generation: Towards a Reference Framework

Nathalia Nascimento, Everton Guimaraes, Paulo Alencar

机构 * The Pennsylvania State University（宾夕法尼亚州立大学）； University of Waterloo（多伦多大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 5 pages

2510.03795 2025-10-07 cs.IR cs.CL 85%

Investigating LLM Variability in Personalized Conversational Information Retrieval

Simon Lupart, Daniël van Dijk, Eric Langezaal, Ian van Dort, Mohammad Aliannejadi

机构 * University of Amsterdam（阿姆斯特丹大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 11 pages, 5 figures, SIGIR-AP'25 Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP 2025), December 7--10, 2025, Xi'an, China

1. 评测与基准 32459 篇

When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation

FlexiDataGen: An Adaptive LLM Framework for Dynamic Semantic Dataset Generation in Sensitive Domains

SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks

NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks

Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability

Towards Mining Effective Pedagogical Strategies from Learner-LLM Educational Dialogues

Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

An Empirical Study on LLM-based Agents for Automated Bug Fixing

SARHAchat: An LLM-Based Chatbot for Sexual and Reproductive Health Counseling

A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling

Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores

Pluto: A Benchmark for Evaluating Efficiency of LLM-generated Hardware Code

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge

Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content

SWE-Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering

Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples

Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting

BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology

All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generations

MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automation

LLM-Driven Rubric-Based Assessment of Algebraic Competence in Multi-Stage Block Coding Tasks with Design and Field Evaluation

CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generation

Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models

Evaluating The Impact of Stimulus Quality in Investigations of LLM Language Performance

Exploring Gaps in the APS: Direct Minimal Pair Analysis in LLM Syntactic Assessments

Ads that Talk Back: Implications and Perceptions of Injecting Personalized Advertising into LLM Chatbots

AutoPentester: An LLM Agent-based Framework for Automated Pentesting

NLD-LLM: A systematic framework for evaluating small language transformer models on natural language description

Designing Empirical Studies on LLM-Based Code Generation: Towards a Reference Framework

Investigating LLM Variability in Personalized Conversational Information Retrieval