语言大模型 / LLM

Do Large Language Models Judge Error Severity Like Humans?

Diege Sun, Guanyi Chen, Zhao Fan, Xiaorong Cheng, Tingting He

机构 * School of Psychology（心理学系） ； Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning（湖北省人工智能与智能学习重点实验室） ； National Language Resources Monitor and Research Center for Network Media（网络媒体语言资源监测与研究中心） ； School of Computer Science（计算机科学学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

SafeLawBench: Towards Safe Alignment of Large Language Models

Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Yinyu Wu, Juntao Dai, Yaodong Yang, Sirui Han, Yike Guo

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Accepted to ACL2025 Findings

Benchmarking Large Language Models on Homework Assessment in Circuit Analysis

Liangliang Chen, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models

Yingchaojie Feng, Zhizhang Chen, Zhining Kang, Sijia Wang, Haoyu Tian, Wei Zhang, Minfeng Zhu, Wei Chen

机构 * State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD与计算机图形学国家重点实验室） ； Laboratory of Art and Archaeology Image (Zhejiang University), Ministry of Education（浙江大学艺术与考古图像实验室（教育部）） ； Hangzhou City University（杭州市大学） ； Zhejiang University（浙江大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

ELEVATE-GenAI: Reporting Guidelines for the Use of Large Language Models in Health Economics and Outcomes Research: an ISPOR Working Group on Generative AI Report

Rachael L. Fleurence, Dalia Dawoud, Jiang Bian, Mitchell K. Higashi, Xiaoyan Wang, Hua Xu, Jagpreet Chhatwal, Turgay Ayer

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments 4 Tables, 1 Figure, Supplemental Material

DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis

Prashanth Vijayaraghavan, Soroush Vosoughi, Lamogha Chiazor, Raya Horesh, Rogerio Abreu de Paula, Ehsan Degan, Vandana Mukherjee

机构 * IBM Research, San Jose, CA, USA（IBM研究院，美国加利福尼亚州圣何塞） ； Dartmouth College, Hanover, NH, USA（达特茅斯学院，美国新罕布什尔州汉诺威） ； IBM Research, Hursley, Winchester, UK（IBM研究院，英国赫里福德郡温切斯特） ； IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA（IBM托马斯·J·沃森研究中心，美国纽约州亚当斯） ； IBM Research, Sao Paulo, Brazil（IBM研究院，巴西圣保罗） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 7 (content pages) + 2 (reference pages) + 5 (Appendix pages), 5 figures, 6 Tables, IJCAI 2025

A Survey on Data Contamination for Large Language Models

Yuxing Cheng, Yi Chang, Yuan Wu

机构 * College of Software, Jilin University（吉林大学软件学院） ； School of Artificial Intelligence, Jilin University（吉林大学人工智能学院） ； Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China（知识驱动人机智能工程研究中心，教育部，中国） ； International Center of Future Science, Jilin University（未来科学国际中心，吉林大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktales

Ayuto Tsutsumi, Yuu Jinnai

机构 * Tokyo Metropolitan University（东京 Metropolitan 大学） ； CyberAgent 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Reflection-Bench: Evaluating Epistemic Agency in Large Language Models

Lingyu Li, Yixu Wang, Haiquan Zhao, Shuqi Kong, Yan Teng, Chunbo Li, Yingchun Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.AI

Comments 29 pages, 19 figures, 9 tables

Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection

Yachao Zhao, Bo Wang, Yan Wang, Dongming Zhao, Ruifang He, Yuexian Hou

机构 * College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学） ； AI Lab, China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Accepted by ACL 2025

BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks

Anna Sokol, Elizabeth Daly, Michael Hind, David Piorkowski, Xiangliang Zhang, Nuno Moniz, Nitesh Chawla

机构 * University of Notre Dame（诺特尔大学） ； IBM Research Ireland（IBM爱尔兰研究院） ； IBM Research NY, USA（IBM纽约研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Large Language Model Evaluation via Matrix Nuclear-Norm

Yahan Li, Tingyu Xia, Yi Chang, Yuan Wu

机构 * School of Artificial Intelligence, Jilin University（吉林大学人工智能学院） ； Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University（吉林大学符号计算与知识工程重点实验室） ； International Center of Future Science, Jilin University（吉林大学未来科学国际中心） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 21 pages

A Survey on Employing Large Language Models for Text-to-SQL Tasks

Liang Shi, Zhengju Tang, Nan Zhang, Xiaotong Zhang, Zhi Yang

机构 * School of Computer Science, Peking University（北京大学计算机科学系） ； SINGDATA CLOUD PTE. LTD（SINGDATA CLOUD 股份有限公司） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by ACM Computing Surveys (CSUR)

Unified Large Language Models for Misinformation Detection in Low-Resource Linguistic Settings

Muhammad Islam, Javed Ali Khan, Mohammed Abaker, Ali Daud, Azeem Irshad

机构 * College of Science and Engineering（科学与工程学院） ； Department of Computer Science（计算机科学系） ； University of Hertfordshire（赫特福德郡大学） ； Applied College, King Khalid University（国王·卡利德大学应用学院） ； Faculty of Resilience（韧性学院） ； Faculty of Computer Science（计算机科学系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge

Md Tahmid Rahman Laskar, Israt Jahan, Elham Dolatabadi, Chun Peng, Enamul Hoque, Jimmy Huang

机构 * York University（约克大学） ； Vector Institute Toronto（多伦多向量研究所） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted at ACL 2025 (Main Conference)

Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study

Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank, Matthias Assenmacher

机构 * LMU Munich（慕尼黑大学） ； Munich Center for Machine Learning（慕尼黑机器学习中心） ； University of Maryland, College Park（马里兰大学学院市分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL 2025

Calibration of Large Language Models on Code Summarization

Yuvraj Virk, Premkumar Devanbu, Toufique Ahmed

机构 * UC Davis（加州大学戴维斯分校） ； IBM Research（IBM研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Large Language Model-Driven Distributed Integrated Multimodal Sensing and Semantic Communications

Yubo Peng, Luping Xiang, Bingxin Zhang, Kun Yang

机构 * State Key Laboratory of Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学） ； School of Intelligent Software and Engineering, Nanjing University (Suzhou Campus)（智能软件与工程学院，南京大学（苏州校区）） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities

Yiming Huang, Jiyu Guo, Wenxin Mao, Cuiyun Gao, Peiyi Han, Chuanyi Liu, Qing Ling

机构 * Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）） ； Peng Cheng Laboratory（鹏城实验室） ； Sun Yat-sen University（中山大学） ； Great Bay University（广东医科大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Submitted to ACM Computing Surveys (CSUR). Currently under review

Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons

Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh, Jin Vivian Lee, Daniel Alexander Alber, Karl L. Sangwon, Douglas Kondziolka, Eric Karl Oermann

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 22 pages, 3 main figures, 3 supplemental figures

Augment or Not? A Comparative Study of Pure and Augmented Large Language Model Recommenders

Wei-Hsiang Huang, Chen-Wei Ke, Wei-Ning Chiu, Yu-Xuan Su, Chun-Chun Yang, Chieh-Yuan Cheng, Yun-Nung Chen, Pu-Jen Cheng

机构 * National Taiwan University（台湾大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

A Survey of Uncertainty Estimation Methods on Large Language Models

Zhiqiu Xia, Jinxuan Xu, Yuqian Zhang, Hang Liu

机构 * Rutgers, The State University of New Jersey（新泽西州立大学拉特格斯分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL Findings 2025

Assessment of L2 Oral Proficiency using Speech Large Language Models

Rao Ma, Mengjie Qian, Siyuan Tang, Stefano Bannò, Kate M. Knill, Mark J. F. Gales

机构 * University of Cambridge（剑桥大学） ； ALTA Institute（ALTA研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments submitted to Interspeech

Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders

Boyi Deng, Yu Wan, Yidan Zhang, Baosong Yang, Fuli Feng

机构 * University of Science and Technology of China（中国科学技术大学） ； Sichuan University（四川大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL 2025 main

Knowledge Boundary of Large Language Models: A Survey

Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, Yang Deng

机构 * National University of Singapore（国立新加坡大学） ； Singapore Management University（新加坡管理大学） ； Singapore University of Technology and Design（新加坡科技设计大学） ； The Chinese University of Hong Kong（香港中文大学） ； University of Minnesota - Twin Cities（明尼苏达大学双城分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 (main)

Evaluating Large Language Models for Code Review

Umut Cihan, Arda İçöz, Vahid Haratian, Eray Tüzün

机构 * Bilkent University（比尔肯特大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Large Language Models in Code Co-generation for Safe Autonomous Vehicles

Ali Nouri, Beatriz Cabrero-Daniel, Zhennan Fei, Krishna Ronanki, Håkan Sivencrona, Christian Berger

机构 * Volvo Cars（沃尔沃汽车公司） ； University of Gothenburg（哥德堡大学） ； Chalmers University of Technology（查尔姆斯理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments Accepted in the 44th International Conference on Computer Safety, Reliability and Security (SafeComp 2025)

ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models

Yachuan Liu, Xiaochun Wei, Lin Shi, Xinnuo Li, Bohan Zhang, Paramveer Dhillon, Qiaozhu Mei

机构 * University of Michigan（密歇根大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.LG

Judging It, Washing It: Scoring and Greenwashing Corporate Climate Disclosures using Large Language Models

Marianne Chuang, Gabriel Chuang, Cheryl Chuang, John Chuang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 17 pages, 12 figures. To appear, ClimateNLP 2025