语言大模型 / LLM

B-score: Detecting biases in large language models using response history

An Vo, Mohammad Reza Taesiri, Daeyoung Kim, Anh Totti Nguyen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICML 2025 (Main track)

CASTILLO: Characterizing Response Length Distributions of Large Language Models

Daniel F. Perez-Ramirez, Dejan Kostic, Magnus Boman

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Dataset available in https://huggingface.co/datasets/danfperam/castillo and code is available in https://github.com/DanielFPerez/castillo

A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization

Ziqing Wang, Kexin Zhang, Zihan Zhao, Yibo Wen, Abhishek Pandey, Han Liu, Kaize Ding

机构 * Northwestern University（西北大学） ； AbbVie（阿比维） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Under review

BenCzechMark : A Czech-centric Multitask and Multimetric Benchmark for Large Language Models with Duel Scoring Mechanism

Martin Fajcik, Martin Docekal, Jan Dolezal, Karel Ondrej, Karel Beneš, Jan Kapsa, Pavel Smrz, Alexander Polok, Michal Hradis, Zuzana Neverilova, Ales Horak, Radoslav Sabol, Michal Stefanik, Adam Jirkovsky, David Adamczyk, Petr Hyner, Jan Hula, Hynek Kydlicek

机构 * Brno University of Technology（布拉格技术大学） ； Masaryk University（马萨里大学） ； University of Helsinki（赫尔辛基大学） ； Czech Technical University in Prague（布拉格捷克技术大学） ； University of Ostrava（奥斯特拉瓦大学） ； CIIRC（捷克信息与通信研究中心） ； Hugging Face 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Accepted to TACL

Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering

Yuan Sui, Yufei He, Zifeng Ding, Bryan Hooi

机构 * National University of Singapore（新加坡国立大学） ； University of Cambridge（剑桥大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted by ACL 2025

Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme One

Yiwen Song, Qianyue Hao, Qingmin Liao, Jian Yuan, Yong Li

机构 * Department of Electronic Engineering, BNRist, Tsinghua University（电子工程系、BNRist、清华大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

A Comparative Study of Large Language Models and Human Personality Traits

Wang Jiaqi, Wang bo, Guo fa, Cheng cheng, Yang li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Role-Playing Evaluation for Large Language Models

Yassine El Boudouri, Walter Nuninger, Julian Alvarez, Yvan Peter

机构 * Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL（里尔大学、国家科学研究中心、里尔中央理工大学、UMR 9189 CRIStAL） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

ARS: Automatic Routing Solver with Large Language Models

Kai Li, Fei Liu, Zhenkun Wang, Xialiang Tong, Xiongwei Han, Mingxuan Yuan, Qingfu Zhang

机构 * Southern University of Science and Technology（南方科技大学） ； City University of Hong Kong（香港城市大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Authorship is under discussion; arXiv release will follow finalization

Decoding the Mind of Large Language Models: A Quantitative Evaluation of Ideology and Biases

Manari Hirose, Masato Uchida

机构 * Waseda University（早稻田大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 5 figures, 17 tables

Journal ref 2025 International Joint Conference on Neural Networks (IJCNN 2025)

Personalized Author Obfuscation with Large Language Models

Mohammad Shokri, Sarah Ita Levitan, Rivka Levitan

机构 * The Graduate Center CUNY（纽约大学研究生中心） ； Hunter College CUNY（纽约大学亨特学院） ； Brooklyn College CUNY（纽约大学布鲁克林学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models

Abdullah Mushtaq, Imran Taj, Rafay Naeem, Ibrahim Ghaznavi, Junaid Qadir

机构 * Information Technology University Department of Computer Science（信息科技大学计算机科学系） ； Zayed University College of Interdisciplinary Studies（扎耶德大学跨学科研究学院） ； Computer Science and Engineering Department（计算机科学与工程系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Preprint. Submitted to the Journal of Artificial Intelligence Research (JAIR) on April 29, 2025

Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence

Timothy R. McIntosh, Teo Susnjak, Nalin Arachchilage, Tong Liu, Paul Watters, Malka N. Halgamuge

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Re-evaluating Open-ended Evaluation of Large Language Models

Siqi Liu, Ian Gemp, Luke Marris, Georgios Piliouras, Nicolas Heess, Marc Lanctot

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Published at ICLR 2025

UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models

Yu Zheng, Longyi Liu, Yuming Lin, Jie Feng, Guozhen Zhang, Depeng Jin, Yong Li

机构 * The Thørväld Group（Thørväld集团） ； Inria Paris-Rocquencourt（Inria巴黎-罗克桑court分部） ； Rajiv Gandhi University Doimukh（拉贾格恩迪大学多伊穆克） ； Tsinghua University（清华大学） ； Palmer Research Laboratories（帕勒默研究实验室） ； The Kumquat Consortium（Kumquat联盟） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework

Yuan Xia, Akanksha Atrey, Fadoua Khmaissia, Kedar S. Namjoshi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Semantic Consistency for Assuring Reliability of Large Language Models

Harsh Raj, Vipul Gupta, Domenic Rosati, Subhabrata Majumdar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments An updated version of this preprint is available at arXiv:2502.15924, and has been accepted at the Transactions on Machine Learning Research

Research on Cloud Platform Network Traffic Monitoring and Anomaly Detection System based on Large Language Models

Ze Yang, Yihong Jin, Juntian Liu, Xinhe Xu, Yihan Zhang, Shuyang Ji

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments Proceedings of 2025 IEEE 7th International Conference on Communications, Information System and Computer Engineering (CISCE 2025)

Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmark

Hanlei Zhang, Zhuohang Li, Yeshuang Zhu, Hua Xu, Peiwu Wang, Haige Zhu, Jie Zhou, Jinchao Zhang

机构 * Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； Pattern Recognition Center, WeChat AI, Tencent Inc, China（腾讯人工智能研究院） ； Kennesaw State University（凯斯西储大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 5 figures

How Effective are Generative Large Language Models in Performing Requirements Classification?

Waad Alhoshan, Alessio Ferrari, Liping Zhao

机构 * Imam Mohammad Ibn Saud Islamic University (IMSIU)（伊玛姆·莫哈梅德·伊本·萨乌德伊斯兰大学） ； University College Dublin (UCD)（都柏林大学学院） ； University of Manchester（曼彻斯特大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach

Ruizhe Li, Chiwei Zhu, Benfeng Xu, Xiaorui Wang, Zhendong Mao

机构 * University of Science and Technology of China（中国科学技术大学） ； MetastoneTechnology（metastone技术） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Codenames as a Benchmark for Large Language Models

Matthew Stephenson, Matthew Sidji, Benoît Ronval

机构 * College of Science and Engineering, Flinders University（科学与工程学院，弗林德斯大学） ； Faculty of Engineering and IT, University of Melbourne（工程与信息技术学院，墨尔本大学） ； ICTEAM, UCLouvain（ICTEAM，UCLouvain） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 2 figures, 2 tables

HealthGenie: Empowering Users with Healthy Dietary Guidance through Knowledge Graph and Large Language Models

Fan Gao, Xinjie Zhao, Ding Xia, Zhongyi Zhou, Rui Yang, Jinghui Lu, Hang Jiang, Chanjun Park, Irene Li

机构 * The University of Tokyo（东京大学） ； Duke-NUS Medical School（杜克-新加坡国立大学医学院） ； Smartor Inc.（Smartor公司） ； MIT Media Lab（麻省理工学院媒体实验室） ； Korea University（韩国大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension

Xinnan Dai, Haohao Qu, Yifen Shen, Bohang Zhang, Qihao Wen, Wenqi Fan, Dongsheng Li, Jiliang Tang, Caihua Shan

机构 * Michigan State University（密歇根州立大学） ； The Hong Kong Polytechnic University（香港理工大学） ； Microsoft Research（微软研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments The paper is published in ICLR 2025

CPG-EVAL: A Multi-Tiered Benchmark for Evaluating the Chinese Pedagogical Grammar Competence of Large Language Models

Dong Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 1 figure, 3 tables

You've Changed: Detecting Modification of Black-Box Large Language Models

Alden Dima, James Foulds, Shimei Pan, Philip Feldman

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 4 figures

Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability

Jennifer Haase, Paul H. P. Hanel, Sebastian Pokutta

专题命中 评测与基准 ：LLM(title,abstract)；language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 19 pages + Appendix, 13 figure

On the attribution of confidence to large language models

Geoff Keeling, Winnie Street

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 0 figures

Balancing Rigor and Utility: Mitigating Cognitive Biases in Large Language Models for Multiple-Choice Questions

Hanyang Zhong, Liman Wang, Wenting Cao, Zeyuan Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments This work has been accepted as a full paper at the 2025 Annual Conference of the Cognitive Science Society (CogSci 2025) and will be presented in the form of a poster. The dataset and project website are available at: https://hanyangzhong.github.io/BRU-website/

AGENT: An Aerial Vehicle Generation and Design Tool Using Large Language Models

Colin Samplawski, Adam D. Cobb, Susmit Jha

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG