语言大模型 / LLM - arXivDaily 专题

2508.06851 2025-08-12 cs.AI cs.CY 88%

MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams

Pengfei Zhou, Xiaopeng Peng, Fanrui Zhang, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Zekai Li, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang

机构 * Shanghai AI Laboratory（上海人工智能实验室）； Shanghai Innovation Institute（上海创新研究院）； USTC（中国科学技术大学）； RIT（罗切斯特理工学院）； HIT（哈尔滨工业大学）； WHU（武汉大学）； MBZUAI（马克斯·普朗克人工智能研究所）； NUS（新加坡国立大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 35 pages, 33 figures

2508.06155 2025-08-11 cs.CL 88%

Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach

Renhan Zhang, Lian Lian, Zhen Qi, Guiran Liu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2508.06145 2025-08-11 cs.AI 88%

Retrieval Augmented Large Language Model System for Comprehensive Drug Contraindications

Byeonghun Bang, Jongsuk Yoon, Dong-Jin Chang, Seho Park, Yong Oh Lee

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2508.05083 2025-08-08 cs.AI 88%

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

Dexuan Xu, Jieyi Wang, Zhongyan Chai, Yongzhi Cao, Hanpin Wang, Huamin Zhang, Yu Huang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 18 pages

2507.02088 2025-08-08 cs.CL 88%

McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models

Tian Lan, Xiangdong Su, Xu Liu, Ruirui Wang, Ke Chang, Jiang Li, Guanglai Gao

机构 * College of Computer Science, Inner Mongolia University（内蒙古大学计算机科学学院）； National & Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian（蒙古语智能信息处理技术国家与地方联合工程研究中心）； Inner Mongolia Key Laboratory of Multilingual Artificial Intelligence Technology（内蒙古多语种人工智能技术重点实验室）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL2025 Findings

Journal ref In Findings of the Association for Computational Linguistics: ACL 2025, pages 6033-6056, Vienna, Austria. Association for Computational Linguistics

2410.17519 2025-08-08 cs.CL 88%

Large Language Models Still Exhibit Bias in Long Text

Wonje Jeung, Dongjae Jeon, Ashkan Yousefpour, Jonghyun Choi

机构 * Yonsei University（延世大学）； Seoul National University（首尔国立大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL, code and models are available at https://github.com/WonjeJeung/LTF-TEST

2508.01006 2025-08-05 cs.CL 88%

UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in Urdu

Farah Adeeba, Brian Dillon, Hassan Sajjad, Rajesh Bhatt

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2505.16326 2025-08-05 cs.LG 88%

ChemMLLM: Chemical Multimodal Large Language Model

Qian Tan, Dongzhan Zhou, Peng Xia, Wanhao Liu, Wanli Ouyang, Lei Bai, Yuqiang Li, Tianfan Fu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments 23 pages

2502.19166 2025-08-05 cs.SE cs.LG 88%

CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation

Kaiwen Yan, Hongcheng Guo, Xuanqing Shi, Shaosheng Cao, Donglin Di, Zhoujun Li

机构 * Beihang University（北航大学）； Tsinghua University（清华大学）； Xiaohongshu（小红书）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments Accepted as an ACL 2025 Industry Track paper (15 pages)

2507.22542 2025-07-31 cs.CL 88%

A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support

Long S. T. Nguyen, Truong P. Hua, Thanh M. Nguyen, Toan Q. Pham, Nam K. Ngo, An X. Nguyen, Nghi D. M. Pham, Nghia H. Nguyen, Tho T. Quan

机构 * URA Research Group, Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh City, Vietnam（乌尔拉研究组，计算机科学与工程学院，胡志明城理工大学（HCMUT），胡志明城，越南）； Vietnam National University Ho Chi Minh City, Ho Chi Minh City, Vietnam（越南国家大学胡志明城分校，胡志明城，越南）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Under review at ICCCI 2025

2505.19959 2025-07-31 cs.CL 88%

MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models

Zhongzhan Huang, Guoming Ling, Shanshan Zhong, Hefeng Wu, Liang Lin

机构 * Sun Yat-sen University（中山大学）； Peng Cheng Laboratory（鹏城实验室）； Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL'25 main track

2408.07846 2025-07-31 cs.SE cs.AI 88%

A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites

Andrea Lops, Fedelucio Narducci, Azzurra Ragone, Michelantonio Trizio, Claudio Bartolini

机构 * Polytechnic University of Bari \& Wideverse Bari, Italy ； Polytechnic University of Bari Bari, Italy ； Independent Researcher ； University of Bari Bari, Italy

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Journal ref 2025 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

2507.21773 2025-07-30 cs.CL 88%

AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models

Lian Yan, Haotian Wang, Chen Tang, Haifeng Liu, Tianyang Sun, Liangliang Liu, Yi Guan, Jingchi Jiang

机构 * Harbin Institute of Technology（哈尔滨工业大学）； MemTensor (Shanghai) Technology Co., Ltd.（MemTensor（上海）技术有限公司）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 36 pages, 22 figures

2507.21133 2025-07-30 cs.CR cs.AI 88%

Analysis of Threat-Based Manipulation in Large Language Models: A Dual Perspective on Vulnerabilities and Performance Enhancement Opportunities

Atil Samancioglu

机构 * Atil Samancioglu（独立研究者）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2507.19185 2025-07-28 cs.CR cs.AI 88%

PrompTrend: Continuous Community-Driven Vulnerability Discovery and Assessment for Large Language Models

Tarek Gasmi, Ramzi Guesmi, Mootez Aloui, Jihene Bennaceur

机构 * University of Manouba（曼努巴大学）； University of Jendouba（杰努布大学）； LETI Laboratory, University of Sfax（萨克斯大学LETI实验室）； DataDoIt（DataDoIt公司）； South Mediterranean University（南地中海大学）

专题命中评测与基准：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.AI

2507.14900 2025-07-24 cs.CL 88%

From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment

Chongxuan Huang, Yongshi Ye, Biao Fu, Qifeng Su, Xiaodong Shi

机构 * School of Informatics, Xiamen University（厦门大学信息学院）； Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能学院）； Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism（福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室（厦门大学），文化和旅游部）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ACL main 2025

2507.15224 2025-07-22 cs.SE cs.AI 88%

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation

Yibo He, Shuoran Zhao, Jiaming Huang, Yingjie Fu, Hao Yu, Cunjian Huang, Tao Xie

机构 * Peking University（北京大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； The Hong Kong University of Science and Technology（香港科学与技术大学）； DAMO Academy, Alibaba Group（阿里云达摩院）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2507.13334 2025-07-22 cs.CL 88%

A Survey of Context Engineering for Large Language Models

Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, Shenghua Liu

机构 * Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）； University of California, Merced（加州大学默塞德分校）； The University of Queensland（昆士兰大学）； Peking University（北京大学）； Tsinghua University（清华大学）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ongoing work; 166 pages, 1411 citations

2503.16148 2025-07-22 cs.CY cs.CL 88%

Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models

Mats Faulborn, Indira Sen, Max Pellert, Andreas Spitz, David Garcia

机构 * scieneers GmbH（scieneers公司）； University of Mannheim（曼海姆大学）； Barcelona Super Computing Center（巴塞罗那超级计算中心）； University of Konstanz（康斯坦茨大学）

专题命中评测与基准：language model(title,abstract)；large language model(title)；prompting(abstract)；分类 cs.CL

Comments Preprint of ACL 2025 paper

2507.14586 2025-07-22 physics.app-ph cs.CE cs.CL 88%

What do Large Language Models know about materials?

Adrian Ehrenhofer, Thomas Wallmersperger, Gianaurelio Cuniberti

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2507.11882 2025-07-17 cs.CL 88%

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models

Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang

机构 * Alibaba International Digital Commerce（阿里巴巴国际数字商务）； University of Aberdeen（阿伯丁大学）； MBZUAI

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ACL 2025 Main Conference paper

2507.09583 2025-07-15 cs.SE cs.AI 88%

A Serverless Architecture for Real-Time Stock Analysis using Large Language Models: An Iterative Development and Debugging Case Study

Taniv Ashraf

机构 * Independent Researcher（独立研究者）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 6 pages. The live application can be viewed at https://codepen.io/tanivashraf/pen/GgpgxBY and the source code is available at https://github.com/TanivAshraf/ai-stock-analyzer

2507.09090 2025-07-15 cs.IR cs.CL 88%

DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate

Anthony Miyaguchi, Conor Johnston, Aaryan Potdar

机构 * Georgia Institute of Technology（佐治亚理工学院）； Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学）； Joint Institute for Nuclear Research（联合核研究所）； Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）； University of Skövde（斯德哥尔摩大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2404.12464 2025-07-10 cs.CL 88%

NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models

Abhinav Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, Maarten Sap

机构 * Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）； Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Published at NAACL 2025, Albuquerque, New Mexico, USA

Journal ref Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) 2373-2403

2507.06157 2025-07-09 cs.RO cs.CL 88%

Evaluation of Habitat Robotics using Large Language Models

William Li, Lei Hamilton, Kaise Al-natour, Sanjeev Mohindra

机构 * Artificial Intelligence Group MIT Lincoln Lab（人工智能组 MIT 林肯实验室）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 6 pages, IEEE HPEC submission

2507.04976 2025-07-08 cs.CV cs.CL 88%

Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models

Eunseop Yoon, Hee Suk Yoon, Mark A. Hasegawa-Johnson, Chang D. Yoo

机构 * Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）； University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ICLR 2025

2507.04854 2025-07-08 cs.CL 88%

$\textit{Grahak-Nyay:}$ Consumer Grievance Redressal through Large Language Models

Shrey Ganatra, Swapnil Bhattacharyya, Harshvivek Kashid, Spandan Anaokar, Shruti Nair, Reshma Sekhar, Siddharth Manohar, Rahul Hemrajani, Pushpak Bhattacharyya

机构 * Indian Institute of Technology Bombay（印度理工学院孟买学院）； National Law School of India University（印度国家法律学校大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2409.09318 2025-07-08 cs.CL cs.CV 88%

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

Yahan Tu, Rui Hu, Jitao Sang

机构 * Beijing Jiaotong University（北京交通大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2406.15695 2025-07-08 cs.CL 88%

SS-GEN: A Social Story Generation Framework with Large Language Models

Yi Feng, Mingyang Song, Jiaqi Wang, Zhuang Chen, Guanqun Bi, Minlie Huang, Liping Jing, Jian Yu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments AAAI 2025 (Oral)

2505.19121 2025-07-03 cs.CL 88%

Delving into Multilingual Ethical Bias: The MSQAD with Statistical Hypothesis Tests for Large Language Models

Seunguk Yu, Juhwan Choi, Youngbin Kim

机构 * Chung-Ang University（Chung-Ang 大学）； AITRICS（AITRICS 研究所）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ACL 2025 main conference

1. 评测与基准 32207 篇

MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams

Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach

Retrieval Augmented Large Language Model System for Comprehensive Drug Contraindications

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models

Large Language Models Still Exhibit Bias in Long Text

UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in Urdu

ChemMLLM: Chemical Multimodal Large Language Model

CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation

A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support

MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models

A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites

AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models

Analysis of Threat-Based Manipulation in Large Language Models: A Dual Perspective on Vulnerabilities and Performance Enhancement Opportunities

PrompTrend: Continuous Community-Driven Vulnerability Discovery and Assessment for Large Language Models

From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation

A Survey of Context Engineering for Large Language Models

Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models

What do Large Language Models know about materials?

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models

A Serverless Architecture for Real-Time Stock Analysis using Large Language Models: An Iterative Development and Debugging Case Study

DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate

NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models

Evaluation of Habitat Robotics using Large Language Models

Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models

$\textit{Grahak-Nyay:}$ Consumer Grievance Redressal through Large Language Models

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

SS-GEN: A Social Story Generation Framework with Large Language Models

Delving into Multilingual Ethical Bias: The MSQAD with Statistical Hypothesis Tests for Large Language Models