语言大模型 / LLM

Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering

Bowen Jiang, Runchuan Zhu, Jiang Wu, Zinco Jiang, Yifan He, Junyuan Gao, Jia Yu, Rui Min, Yinfan Wang, Haote Yang, Songyang Zhang, Dahua Lin, Lijun Wu, Conghui He

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Peking University（北京大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； Shanghai University（上海大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Equal contribution: Bowen Jiang, Runchuan Zhu, Jiang Wu; Corresponding author: Conghui He

Evolutionary Computation and Large Language Models: A Survey of Methods, Synergies, and Applications

Dikshit Chauhan, Bapi Dutta, Indu Bala, Niki van Stein, Thomas Bäck, Anupam Yadav

机构 * Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电气与计算机工程系） ； Department of Computer Science, Universidad de Jaén（西班牙Jaén大学计算机科学系） ； School of Computer and Mathematical Sciences, University of Adelaide（阿德莱德大学计算机与数学科学学院） ； Leiden Institute of Advanced Computer Science, University Leiden（莱顿大学先进计算机科学研究所） ； Department of Mathematics and Computing, Dr. B. R. Ambedkar National Institute of Technology（德拉·B·R·阿姆贝卡尔国立理工学院数学与计算系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation

Dun Yuan, Hao Zhou, Di Wu, Xue Liu, Hao Chen, Yan Xin, Jianzhong, Zhang

机构 * School of Computer Science（计算机科学学院） ； Department of Electrical and Computer Engineering（电气与计算机工程系） ； Standards and Mobility Innovation Lab（标准与移动创新实验室） ； Samsung Research America（三星美国研究） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments This work has been accepted to ICC 2025 IEEE International Conference on Communications. copyright 2025 IEEE

Can Large Language Models Understand Internet Buzzwords Through User-Generated Content

Chen Huang, Junkai Luo, Xinzuo Wang, Wenqiang Lei, Jiancheng Lv

机构 * College of Computer Science, Sichuan University, China（四川大学计算机学院） ； Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China（教育部机器学习与工业智能工程研究中心） ； JD.com, China（京东公司） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL 2025 Main Paper. Our dataset and code are available at https://github.com/SCUNLP/Buzzword

Empowering the Deaf and Hard of Hearing Community: Enhancing Video Captions Using Large Language Models

Nadeen Fathallah, Monika Bhole, Steffen Staab

机构 * Analytic Computing, Institute for Artificial Intelligence, University of Stuttgart（分析计算中心、人工智能研究所、斯图加特大学） ； University of Stuttgart（斯图加特大学） ； University of Southampton（南安普顿大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting

He Chang, Chenchen Ye, Zhulin Tao, Jie Wu, Zhengmao Yang, Yunshan Ma, Xianglin Huang, Tat-Seng Chua

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Source framing triggers systematic evaluation bias in Large Language Models

Federico Germani, Giovanni Spitale

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

A Survey of Attacks on Large Language Models

Wenrui Xu, Keshab K. Parhi

机构 * Department of Electrical and Computer Engineering, University of Minnesota（电气与计算机工程系，明尼苏达大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation

Nishat Raihan, Antonios Anastasopoulos, Marcos Zampieri

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 30 Pages

Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective

Yutao Mou, Xiao Deng, Yuxiao Luo, Shikun Zhang, Wei Ye

机构 * National Engineering Research Center for Software Engineering, Peking University, China（软件工程国家工程研究中心，北京大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by ACL2025 Main Conference

ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts

Jing-Cheng Pang, Kaiyuan Li, Yidi Wang, Si-Hang Yang, Shengyi Jiang, Yang Yu

机构 * National Key Laboratory for Novel Software Technology Nanjing University China（南京大学新型软件技术国家重点实验室） ； School of Artificial Intelligence Nanjing University China（南京大学人工智能学院） ； The University of Hong Kong（香港大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

System Log Parsing with Large Language Models: A Review

Viktor Beck, Max Landauer, Markus Wurzenberger, Florian Skopik, Andreas Rauber

机构 * AIT Austrian Institute of Technology（奥地利技术研究院） ； TU Wien（维也纳技术大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments 36 pages, 11 figures

A Survey on Large Language Models in Multimodal Recommender Systems

Alejo Lopez-Avila, Jinhua Du

机构 * Huawei London Research Centre（华为伦敦研究中心） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 30 pages, 6 figures

A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Bias

Brandon Smith, Mohamed Reda Bouadjenek, Tahsin Alamgir Kheya, Phillip Dawson, Sunil Aryal

机构 * Deakin University（德克萨斯大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Federated Large Language Models: Feasibility, Robustness, Security and Future Directions

Wenhao Jiang, Yuchuan Luo, Guilin Deng, Silong Chen, Xu Yang, Shihong Wu, Xinwen Gao, Lin Liu, Shaojing Fu

机构 * National University of Defense Technology（国防科技大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 35 pages

DeepMath-Creative: A Benchmark for Evaluating Mathematical Creativity of Large Language Models

Xiaoyang Chen, Xinan Dai, Yu Du, Qian Feng, Naixu Guo, Tingshuo Gu, Yuting Gao, Yingyi Gao, Xudong Han, Xiang Jiang, Yilin Jin, Hongyi Lin, Shisheng Lin, Xiangnan Li, Yuante Li, Yixing Li, Zhentao Lai, Zilu Ma, Yingrong Peng, Jiacheng Qian, Hao-Yu Sun, Jianbo Sun, Zirui Wang, Siwei Wu, Zian Wang, Bin Xu, Jianghao Xu, Yiyang Yu, Zichuan Yang, Hongji Zha, Ruichong Zhang

机构 * School of Mathematical Sciences, Tongji University（同济大学数学科学学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 14 pages, 4 figures

User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Data

Haowei Yang, Qingyi Lu, Yang Wang, Sibei Liu, Jiayun Zheng, Ao Xiang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

A Proposal for Evaluating the Operational Risk for ChatBots based on Large Language Models

Pedro Pinacho-Davidson, Fernando Gutierrez, Pablo Zapata, Rodolfo Vergara, Pablo Aqueveque

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 21 pages

LogiDebrief: A Signal-Temporal Logic based Automated Debriefing Approach with Large Language Models Integration

Zirong Chen, Ziyan An, Jennifer Reynolds, Kristin Mullen, Stephen Martini, Meiyi Ma

机构 * Department of Computer Science, Vanderbilt University（维斯尼尔大学计算机科学系） ； Metro Nashville Department of Emergency Communications（孟菲斯市紧急通讯部门） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments Accepted at IJCAI-2025

Can Large Language Models Predict Parallel Code Performance?

Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk W. Cameron, Gal Oren

机构 * Virginia Tech, USA（弗吉尼亚理工大学） ； LLNL, USA（劳伦斯利弗莫尔国家实验室） ； Code Metal, USA（Code Metal公司） ； Stanford University, Technion, USA（斯坦福大学、技术学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 5 pages, 4 figures, accepted to AI4Sys Workshop at HPDC 2025

Validating the Effectiveness of a Large Language Model-based Approach for Identifying Children's Development across Various Free Play Settings in Kindergarten

Yuanyuan Yang, Yuan Shen, Tianchen Sun, Yangbin Xie

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 15 pages, 4 figures

Large Language Model with Region-guided Referring and Grounding for CT Report Generation

Zhixuan Chen, Yequan Bie, Haibo Jin, Hao Chen

机构 * Hong Kong University of Science and Technology（香港科技大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 10 pages

Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning

Shaun Baek, Shaun Esua-Mensah, Cyrus Tsui, Sejan Vigneswaralingam, Abdullah Alali, Michael Lu, Vasu Sharma, Sean O'Brien, Kevin Zhu

机构 * Emory University（埃默里大学） ； Algoverse AI Research（Algoverse AI研究） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

David Jin, Qian Fu, Yuekang Li

机构 * UNSW Sydney（新南威尔士大学悉尼分校） ； Commonwealth Scientific and Industrial Research Organisation (CSIRO) Data61（澳大利亚联邦科学与工业研究组织（CSIRO）Data61） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Do We Need a Detailed Rubric for Automated Essay Scoring using Large Language Models?

Lui Yoshida

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted in AIED 2025. This preprint has not undergone any post-submission improvements or corrections

A Framework to Assess the Persuasion Risks Large Language Model Chatbots Pose to Democratic Societies

Zhongren Chen, Joshua Kalla, Quan Le, Shinpei Nakamura-Sakai, Jasjeet Sekhon, Ruixiao Wang

机构 * Yale University（耶鲁大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese

Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, Yining Hua

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州）） ； Peking University（北京大学） ； Mindverse AI ； Alibaba Group（阿里巴巴集团） ； Zhejiang University（浙江大学） ； Zhejiang University of Technology（浙江工业大学） ； MBZUAI ； NIO（蔚来汽车） ； HSBC（汇丰银行） ； Harvard T.H. Chan School of Public Health（哈佛大学T.H. Chan公共卫生学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Under Review

Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges

Xiao Xiao, Yu Su, Sijing Zhang, Zhang Chen, Yadong Chen, Tian Liu

机构 * Tencent Hunyuan（腾讯文言） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Enhancing Systematic Reviews with Large Language Models: Using GPT-4 and Kimi

Dandan Chen Kaptur, Yue Huang, Xuejun Ryan Ji, Yanhui Guo, Bradley Kaptur

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 13 pages, Paper presented at the National Council on Measurement in Education (NCME) Conference, Denver, Colorado, in April 2025

PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare

Jose G. Moreno, Jesus Lovon, M'Rick Robin-Charlet, Christine Damase-Michel, Lynda Tamine

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Journal ref Workshop CL4Health @ NAACL 2025, May 2025, Albuquerque, New Mexico, United States