语言大模型 / LLM

Annotation Guidelines-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification

Shiqi Liu, Sannyuya Liu, Lele Sha, Zijie Zeng, Dragan Gasevic, Zhi Liu

机构 * National Engineering Research Center of Educational Big Data, Faculty of Artificial Intelligence in Education, Central China Normal University（国家教育大数据工程研究中心，教育人工智能学院，中央财经大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments The manuscript has been accepted for publication in IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2025.3570775

Journal ref Liu, S., Liu, S., Sha, L., Zeng, Z., Gasevic, D., & Liu, Z. (2025). Annotation Guideline-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification. IEEE Transactions on Learning Technologies

Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models

Zhiyin Tan, Jennifer D'Souza

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication in International Journal on Digital Libraries (IJDL)

Journal ref International Journal on Digital Libraries, vol. 26, no. 4, pp. 23, December 2025

Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models

Christian Hobelsberger, Theresa Winner, Andreas Nawroth, Oliver Mitevski, Anna-Carolina Haensch

机构 * LMU Munich（慕尼黑大学） ； Munich Re（慕尼黑再保险） ； relAI ； MCML ； University of Maryland（马里兰大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models

Yeonjun In, Wonjoong Kim, Kanghoon Yoon, Sungchul Kim, Mehrab Tanjim, Sangwu Park, Kibum Kim, Chanyoung Park

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation

Xiaoyu Zhang, Juan Zhai, Shiqing Ma, Qingshuang Bao, Weipeng Jiang, Qian Wang, Chao Shen, Yang Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 27 pages, 13 figures

Compositional Coordination for Multi-Robot Teams with Large Language Models

Zhehui Huang, Guangyao Shi, Yuwei Wu, Vijay Kumar, Gaurav S. Sukhatme

机构 * Department of Computer Science, University of Southern California（计算机科学系，南加州大学） ； GRASP Lab, University of Pennsylvania（GRASP实验室，宾夕法尼亚大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Comments IEEE International Symposium on Multi-Robot & Multi-Agent Systems (MRS 2025) Oral

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge

Nafis Chowdhury, Moinul Haque, Anika Ahmed, Nazia Tasnim, Md. Istiak Hossain Shihab, Sajjadur Rahman, Farig Sadeque

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 4 pages

SODBench: A Large Language Model Approach to Documenting Spreadsheet Operations

Amila Indika, Igor Molybog

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 14 pages, 5 figures, 4 tables

Towards Reliable Evaluation of Large Language Models for Multilingual and Multimodal E-Commerce Applications

Shuyi Xie, Ziqin Liew, Hailing Zhang, Haibo Zhang, Ling Hu, Zhiqiang Zhou, Shuman Liu, Anxiang Zeng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

"You Are Rejected!": An Empirical Study of Large Language Models Taking Hiring Evaluations

Dingjie Fu, Dianxing Shi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Technical Report, 14 pages, 8 figures

On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?

Mingmeng Geng, Thierry Poibeau

机构 * École Normale Supérieure (ENS) - Université Paris Sciences et Lettres (PSL)（巴黎 sciences et lettres 大学（PSL）- 法国规范大学（ENS）） ； Laboratoire Lattice (CNRS, ENS-PSL, Université Sorbonne Nouvelle)（晶格实验室（CNRS、ENS-PSL、索邦大学）） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

A Use-Case Specific Dataset for Measuring Dimensions of Responsible Performance in LLM-generated Text

Alicia Sagae, Chia-Jung Lee, Sandeep Avula, Brandon Dang, Vanessa Murdock

机构 * AWS Responsible AI（AWS负责任人工智能） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages with 3 figures, to appear in Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM '25)

Quantitative LLM Judges

Aishwarya Sahoo, Jeevana Kruthi Karnuthala, Tushar Parmanand Budhwani, Pranchal Agarwal, Sankaran Vaidyanathan, Alexa Siu, Franck Dernoncourt, Jennifer Healey, Nedim Lipka, Ryan Rossi, Uttaran Bhattacharya, Branislav Kveton

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Adobe Research（Adobe研究） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

LLM-empowered knowledge graph construction: A survey

Haonan Bian

机构 * Xidian University（西安电子科技大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System

Jiale Guo, Suizhi Huang, Mei Li, Dong Huang, Xingsheng Chen, Regina Zhang, Zhijiang Guo, Han Yu, Siu-Ming Yiu, Pietro Lio, Kwok-Yan Lam

机构 * Digital Trust Centre, Nanyang Technological University, Singapore（南洋理工大学数字信任中心） ； College of Computing and Data Science, Nanyang Technological University, Singapore（南洋理工大学计算与数据科学学院） ； The Hong Kong University of Science and Technology, Hong Kong（香港科学与技术大学） ； School of Computer Science, Shanghai Jiao Tong University, Shanghai, China（上海交通大学计算机科学学院） ； School of Computing and Data Science, The University of Hong Kong, Hong Kong（香港大学计算与数据科学学院） ； School of Computer Science and Technology, The University of Cambridge, UK（剑桥大学计算机科学与技术学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 22 pages

Stress-Testing Model Specs Reveals Character Differences among Language Models

Jifan Zhang, Henry Sleight, Andi Peng, John Schulman, Esin Durmus

机构 * Anthropic Fellows Program（Anthropic 合作计划） ； Constellation ； Anthropic ； Thinking Machines Lab（Thinking Machines 实验室） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration

Xiuyuan Chen, Jian Zhao, Yuchen Yuan, Tianle Zhang, Huilin Zhou, Zheng Zhu, Ping Hu, Linghe Kong, Chi Zhang, Weiran Huang, Xuelong Li

机构 * Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信） ； School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院） ； University of Science and Technology of China（中国科学技术大学） ； GigaAI ； School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations

Ernests Lavrinovics, Russa Biswas, Katja Hose, Johannes Bjerva

机构 * Department of Computer Science, Aalborg University（计算机科学系，奥胡斯大学） ； Institute of Logic and Computation, TU Wien（逻辑与计算研究所，维也纳技术大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

REOBench: Benchmarking Robustness of Earth Observation Foundation Models

Xiang Li, Yong Tao, Siyuan Zhang, Siwei Liu, Zhitong Xiong, Chunbo Luo, Lu Liu, Mykola Pechenizkiy, Xiao Xiang Zhu, Tianjin Huang

机构 * University of Bristol, UK（英国布里斯托大学） ； University of Exeter, UK（英国埃克塞特大学） ； South China Normal University, China（华南师范大学） ； The University of Aberdeen, UK（英国阿伯丁大学） ； Technical University of Munich, Germany（慕尼黑技术大学） ； Eindhoven University of Technology, NL（埃因霍温理工大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；language model(abstract)

Comments Accepted to NeruIPS 2025 D&B Track

Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models

Peter Robicheaux, Matvei Popov, Anish Madan, Isaac Robinson, Joseph Nelson, Deva Ramanan, Neehar Peri

机构 * Roboflow ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments The first two authors contributed equally. This work has been accepted to the Neural Information Processing Systems (NeurIPS) 2025 Datasets & Benchmark Track. Project Page: https://rf100-vl.org/

ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases

Ziqian Zhong, Aditi Raghunathan, Nicholas Carlini

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation

Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger, Sepp Hochreiter

机构 * ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz（林茨ELLIS单位和LIT AI实验室，机器学习研究所，林茨约瑟夫·克里格大学） ； NXAI GmbH（NXAI公司） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Preprint, under review

Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistants

Lixiong Qin, Shilong Ou, Miaoxuan Zhang, Jiangning Wei, Yuhang Zhang, Xiaoshuai Song, Yuchen Liu, Mei Wang, Weiran Xu

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Beijing Normal University（北京师范大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 50 pages, 14 figures, 42 tables. NeurIPS 2025 Datasets and Benchmarks Track

A Structured Review and Quantitative Profiling of Public Brain MRI Datasets for Foundation Model Development

Minh Sao Khue Luu, Margaret V. Benedichuk, Ekaterina I. Roppert, Roman M. Kenzhin, Bair N. Tuchinov

专题命中 评测与基准 ：foundation model(title,abstract)

RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing

Hao Xiang, Tianyi Tang, Yang Su, Bowen Yu, An Yang, Fei Huang, Yichang Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Jingren Zhou, Junyang Lin, Le Sun

机构 * Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室） ； Alibaba Group（阿里巴巴集团） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders

Agam Goyal, Vedant Rathi, William Yeh, Yian Wang, Yuen Chen, Hari Sundaram

机构 * Siebel School of Computing and Data Science（计算机与数据科学学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025

XtraGPT: Context-Aware and Controllable Academic Paper Revision

Nuo Chen, Andre Lin HuiKai, Jiaying Wu, Junyi Hou, Zining Zhang, Qian Wang, Xidong Wang, Bingsheng He

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments Preprint. The model report is available at https://arxiv.org/abs/2505.11336v1

CodeFuse-CR-Bench: A Comprehensiveness-aware Benchmark for End-to-End Code Review Evaluation in Python Projects

Hanyang Guo, Xunjin Zheng, Zihan Liao, Hang Yu, Peng DI, Ziyin Zhang, Hong-Ning Dai

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

How Far Have LLMs Come Toward Automated SATD Taxonomy Construction?

Sota Nakashima, Yuta Ishimoto, Masanari Kondo, Tao Xiao, Yasutaka Kamei

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 5 pages, APSEC 2025

A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese

Tin Van Huynh, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen

机构 * Faculty of Information Science and Engineering, University of Information Technology（信息科学与工程学院，信息技术大学） ； Vietnam National University（越南国家大学） 纠错

专题命中 评测与基准 ：language model(title)；分类 cs.CL

Comments Accepted by Expert Systems with Applications