语言大模型 / LLM

ChatNVD: Advancing Cybersecurity Vulnerability Assessment with Large Language Models

Shivansh Chopra, Hussain Ahmad, Diksha Goel, Claudia Szabo

机构 * The University of Adelaide, Australia（澳大利亚阿德莱德大学） ； CSIRO’s Data61, Australia（澳大利亚CSIRO数据61） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

SeedBench: A Multi-task Benchmark for Evaluating Large Language Models in Seed Science

Jie Ying, Zihong Chen, Zhefan Wang, Wanli Jiang, Chenyang Wang, Zhonghang Yuan, Haoyang Su, Huanjun Kong, Fan Yang, Nanqing Dong

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Yazhouwan National Laboratory（Yazhouwan国家实验室） ； Shanghai Innovation Institute（上海创新研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2025

BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, Jun Sun

机构 * Singapore Management University（新加坡管理大学） ； The University of Melbourne（墨尔本大学） ； Fudan University（复旦大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 22 pages

ToLeaP: Rethinking Development of Tool Learning with Large Language Models

Haotian Chen, Zijun Song, Boye Niu, Ke Zhang, Litu Ou, Yaxi Lu, Zhong Zhang, Xin Cong, Yankai Lin, Zhiyuan Liu, Maosong Sun

机构 * Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； Dalian University of Technology（大连理工大学） ； University of Sydney（悉尼大学） ； Waseda University（早稻田大学） ； National University of Singapore（新加坡国立大学） ； Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models

Shi Qiu, Shaoyang Guo, Zhuo-Yang Song, Yunbo Sun, Zeyu Cai, Jiashen Wei, Tianyu Luo, Yixuan Yin, Haoxu Zhang, Yi Hu, Chenyang Wang, Chencheng Tang, Haoling Chang, Qi Liu, Ziheng Zhou, Tianyu Zhang, Jingtian Zhang, Zhangyi Liu, Minghao Li, Yuku Zhang, Boxuan Jing, Xianqi Yin, Yutong Ren, Zizhuo Fu, Jiaming Ji, Weike Wang, Xudong Tian, Anqi Lv, Laifu Man, Jianxiang Li, Feiyu Tao, Qihua Sun, Zhou Liang, Yushu Mu, Zhongxuan Li, Jing-Jun Zhang, Shutao Zhang, Xiaotian Li, Xingqi Xia, Jiawei Lin, Zheyu Shen, Jiahang Chen, Qiuhao Xiong, Binran Wang, Fengyuan Wang, Ziyang Ni, Bohan Zhang, Fan Cui, Changkun Shao, Qing-Hong Cao, Ming-xing Luo, Yaodong Yang, Muhan Zhang, Hua Xing Zhu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 34 pages ,12 figures, 7 tables, latest update in 2025/05/18

EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models

Bohao Xing, Xin Liu, Guoying Zhao, Chengyu Liu, Xiaolan Fu, Heikki Kälviäinen

机构 * Lappeenranta-Lahti University of Technology LUT（拉佩兰塔-拉赫蒂技术大学） ； University of Oulu（奥卢大学） ； Southeast University（东南大学） ； Shanghai Jiao Tong University（上海交通大学） ； Brno University of Technology（布拉格技术大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values

Martín Rodríguez, Gustavo Rossi, Alejandro Fernandez

机构 * LIFIA, Faculty of Informatics, UNLP, La Plata, Argentina（LIFIA信息学院，UNLP，拉普拉塔，阿根廷） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Under revision at Jornadas de Cloud Computing, Big Data & Emerging Topics (JCC-BD&ET) - 2025

HealthBench: Evaluating Large Language Models Towards Improved Human Health

Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, Karan Singhal

机构 * OpenAI 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Blog: https://openai.com/index/healthbench/ Code: https://github.com/openai/simple-evals

Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies

Xiaoliang Luo, Xinyi Xu, Michael Ramscar, Bradley C. Love

机构 * EmpiriQal Inc.（EmpiriQal公司） ； Department of Mathematics London School of Economics and Political Science（伦敦政治经济学院数学系） ； Department of Psychology University of Tübingen（图宾根大学心理学系） ； Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Bridging Large Language Models and Single-Cell Transcriptomics in Dissecting Selective Motor Neuron Vulnerability

Douglas Jiang, Zilin Dai, Luxuan Zhang, Qiyi Yu, Haoqi Sun, Feng Tian

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

English Please: Evaluating Machine Translation with Large Language Models for Multilingual Bug Reports

Avinash Patil, Siru Tao, Aryan Jadon

机构 * Juniper Networks Inc.（Juniper网络公司） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 8 Pages, 4 Figures, 3 Tables

Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameters

David Exler, Mark Schutera, Markus Reischl, Luca Rettenberger

机构 * Institute for Automation and Applied Informatics（自动化与应用信息研究所） ； Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Promoting Security and Trust on Social Networks: Explainable Cyberbullying Detection Using Large Language Models in a Stream-Based Machine Learning Framework

Silvia García-Méndez, Francisco De Arriba-Pérez

机构 * Information Technologies Group - atlanTTic University of Vigo（信息科技组 - atlanTTic大学维戈分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Journal ref In 11th International Conference on SNAMS (pp. 25-32). IEEE (2024)

AudioBench: A Universal Benchmark for Audio Large Language Models

Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, Nancy F. Chen

机构 * Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I2R），A*STAR，新加坡） ； Centre for Frontier AI Research (CFAR), A*STAR（前沿人工智能研究中心（CFAR），A*STAR） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments v5 - Update acknowledgment; Code: https://github.com/AudioLLMs/AudioBench

A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications

Jian Guan, Junfei Wu, Jia-Nan Li, Chuanqi Cheng, Wei Wu

机构 * Ant Group（蚂蚁集团） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Survey paper; 11 pages; Literature reviewed up to ICLR 2025

SignLLM: Sign Language Production Large Language Models

Sen Fang, Chen Chen, Lei Wang, Ce Zheng, Chunyu Sui, Yapeng Tian

机构 * ASLT 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments website at https://signllm.github.io/

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation

Wenjing Yin, Tianze Sun, Yijiong Yu, Jiawei Fang, Guangyao Su, Jiancheng Wang, Zekun Wang, Wei Wang, Ran Chen, Ziyun Dai, Shuai Yuan, Menghang Dong, Peng Luo, Dong Cao, Da Lei, Yajun Zhang, Hao Chen, Xiang Ma, Yong Liu, Weifeng Liu, Yuanjian Xu, Ji Pei

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Submitted to ACL 2025. Under review

WenyanGPT: A Large Language Model for Classical Chinese Tasks

Xinyu Yao, Mengdi Wang, Bo Chen, Xiaobing Zhao

机构 * Minzu University of China（中国民族大学） ； National Language Resources Monitoring & Research Center of Languages（语言资源监测与研究中心） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Evaluating Large Language Models on Multiword Expressions in Multilingual and Code-Switched Contexts

Frances Laureano De Leon, Harish Tayyar Madabushi, Mark G. Lee

机构 * School of Computer Science University of Birmingham（计算机科学学院 英国伯明翰大学） ； Department of Computer Science University of Bath（计算机科学系 英国巴斯大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Evaluate-and-Purify: Fortifying Code Language Models Against Adversarial Attacks Using LLM-as-a-Judge

Wenhan Mu, Ling Xu, Shuren Pei, Le Mi, Huichi Zhou

机构 * School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件工程学院） ； Imperial College London（伦敦帝国理工学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 25 pages, 6 figures

Pula: Training Large Language Models for Setswana

Nathan Brown, Vukosi Marivate

机构 * Data Science for Social Impact University of Pretoria School of Computing Clemson University（数据科学与社会影响 乌得勒支大学 学术 computing 勃林茅伊大学） ； Data Science for Social Impact Department of Computer Science University of Pretoria（数据科学与社会影响 计算机科学系 乌得勒支大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL

Comments NAACL 2025. 10 pages, 5 tables, 1 figure

Automatically Generating Rules of Malicious Software Packages via Large Language Model

XiangRui Zhang, HaoYu Chen, Yongzhong He, Wenjia Niu, Qiang Li

机构 * School of Cyberspace Security, Beijing Jiaotong University（网络安全学院，北京交通大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 14 pages, 11 figures

Journal ref the 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks(DSN), 2025

The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models

Ronak Pradeep, Nandan Thakur, Shivani Upadhyay, Daniel Campos, Nick Craswell, Jimmy Lin

机构 * University of Waterloo（滑铁卢大学） ； Snowflake ； Microsoft（微软） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments To appear in SIGIR 2025. Significant updates and revisions to arXiv:2411.09607

BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models

Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang, Fei Yuan

机构 * National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家实验室，南京大学） ； University of Zurich（苏黎世大学） ； Carnegie Mellon University（卡内基梅隆大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Can the capability of Large Language Models be described by human ability? A Meta Study

Mingrui Zan, Yunquan Zhang, Boyang Zhang, Fangming Liu, Daning Cheng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts

Jiansheng Li, Xingxuan Zhang, Hao Zou, Yige Guo, Renzhe Xu, Yilong Liu, Chuzhao Zhu, Yue He, Peng Cui

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations

Chitranshu Harbola, Anupam Purwar

专题命中 评测与基准 ：language model(title,abstract)；small language model(title)；LLM(abstract)；分类 cs.CL

ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models

Hengxiang Zhang, Hongfu Gao, Qiang Hu, Guanhua Chen, Lili Yang, Bingyi Jing, Hongxin Wei, Bing Wang, Haifeng Bai, Lei Yang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AdaptRec: A Self-Adaptive Framework for Sequential Recommendations with Large Language Models

Tong Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models

Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL