语言大模型 / LLM

Survey of Specialized Large Language Model

Chenghan Yang, Ruiyu Zhao, Yang Liu, Ling Jiang

机构 * Xiaoduo AI（小多AI） ； Shanghai Jiao Tong University（上海交通大学） ； East China University of Science and Technology（东华大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 1 figures

LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models

Zhiyuan Ning, Tianle Gu, Jiaxin Song, Shixin Hong, Lingyu Li, Huacan Liu, Jie Li, Yixu Wang, Meng Lingyu, Yan Teng, Yingchun Wang

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 7pages, 5 figures

Geopolitical Parallax: Beyond Walter Lippmann Just After Large Language Models

Mehmet Can Yavuz, Humza Gohar Kabir, Aylin Özkan

机构 * Faculty of Engineering and Natural Sciences, Işık University（工程与自然科学学院，伊斯克大学） ； Arky Multimedia（Arky多媒体） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 7 pages, 4 figures, 7 tables

MQAD: A Large-Scale Question Answering Dataset for Training Music Large Language Models

Zhihao Ouyang, Ju-Chiang Wang, Daiyu Zhang, Bin Chen, Shangjie Li, Quan Lin

机构 * ByteDance（字节跳动） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

LongReasonArena: A Long Reasoning Benchmark for Large Language Models

Jiayu Ding, Shuming Ma, Lei Cui, Nanning Zheng, Furu Wei

机构 * IAIR, Xi’an Jiaotong University（IAIR，西安交通大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models

Ke Zhou, Marios Constantinides, Daniele Quercia

机构 * Nokia Bell Labs（诺基亚贝尔实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted in AIES 2025

MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models

Zhongpu Chen, Yinfeng Liu, Long Shi, Xingyan Chen, Yu Zhao, Fuji Ren

机构 * Southwestern University of Finance and Economics（西南财经大学） ； University of Electronic Science and Technology of China（电子科技大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments WWW 2025

ArgCMV: An Argument Summarization Benchmark for the LLM-era

Omkar Gurjar, Agam Goyal, Eshwar Chandrasekharan

机构 * Siebel School of Computing and Data Science（计算与数据科学学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals

Qianli Wang, Van Bach Nguyen, Nils Feldhus, Luis Felipe Villa-Arenas, Christin Seifert, Sebastian Möller, Vera Schmitt

机构 * Quality and Usability Lab, Technische Universität Berlin（柏林技术大学质量与可用性实验室） ； University of Marburg（马尔堡大学） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Deutsche Telekom（德国电信） ； BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at INLG 2025, camera-ready version

Secure Multi-LLM Agentic AI and Agentification for Edge General Intelligence by Zero-Trust: A Survey

Yinqiu Liu, Ruichen Zhang, Haoxiang Luo, Yijing Lin, Geng Sun, Dusit Niyato, Hongyang Du, Zehui Xiong, Yonggang Wen, Abbas Jamalipour, Dong In Kim, Ping Zhang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 35 pages

Tracking World States with Language Models: State-Based Evaluation Using Chess

Romain Harang, Jason Naradowsky, Yaswitha Gujju, Yusuke Miyao

机构 * University of Tokyo（东京大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments Spotlight presentation at ICML 2025 Workshop on Assessing World Models

DemoBias: An Empirical Study to Trace Demographic Biases in Vision Foundation Models

Abu Sufian, Anirudha Ghosh, Debaditya Barman, Marco Leo, Cosimo Distante

机构 * CNR-ISASI（意大利那不勒斯） ； Visva-Bharati（维斯瓦-巴哈蒂） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；language model(abstract)；分类 cs.AI

Comments 6 pages, 4 figures, 13th International Workshop on Biometrics and Forensics (IWBF)

NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision

Xiang Li, Wenyue Hua, Kaijie Zhu, Lingyao Li, Haoyang Ling, Jinkui Chi, Qi Dou, Jindong Wang, Yongfeng Zhang, Xin Ma, Lizhou Fan

机构 * Shandong University, School of Control Science ； Rutgers University, Department of Computer Science 110 Frelinghuysen Road Piscataway New Brunswick NJ USA 08854 ； University of California, Santa Barbara 1210 Cheadle Hall Santa Barbara California USA 93106 ； University of Michigan, School of Information 500 S State St Ann Arbor Michigan USA 48109 ； The Chinese University of Hong Kong, Department of Computer Science ； The College of William \& Mary, School of Computing, Data Sciences \& Physics 200 Stadium Dr Williamsburg Virginia USA 23185 ； The Chinese University of Hong Kong, Department of Psychiatry 9 Chuen On Rd Tai Po New Territories Hong Kong SAR, China 999077 ； Rutgers University, Department of Computer Science ； University of California, Santa Barbara ； University of Michigan, School of Information ； The College of William \& Mary, School of Computing, Data Sciences \& Physics ； The Chinese University of Hong Kong, Department of Psychiatry 纠错

专题命中 评测与基准 ：language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 25 pages, 9 figures, 2 tables

GENIE-ASI: Generative Instruction and Executable Code for Analog Subcircuit Identification

Phuoc Pham, Arun Venkitaraman, Chia-Yu Hsieh, Andrea Bonetti, Stefan Uhlich, Markus Leibl, Simon Hofmann, Eisaku Ohbuchi, Lorenzo Servadei, Ulf Schlichtmann, Robert Wille

机构 * Chair for Design Automation, Technical University of Munich（自动化设计研究所，慕尼黑技术大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Reliable Weak-to-Strong Monitoring of LLM Agents

Neil Kale, Chen Bo Calvin Zhang, Kevin Zhu, Ankit Aich, Paula Rodriguez, Scale Red Team, Christina Q. Knight, Zifan Wang

机构 * Scale AI ； Carnegie Mellon University（卡内基梅隆大学） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 15 figures

Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement

Mohammed Rakibul Hasan, Rafi Majid, Ahanaf Tahmid

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models

Oliver Grainge, Sania Waheed, Jack Stilgoe, Michael Milford, Shoaib Ehsan

专题命中 评测与基准 ：language model(title,abstract)

Comments Accepted to AAAI Fall Symposium 2025 on AI Trustworthiness and Risk Assessment for Challenging Contexts (ATRACC)

Doc2Chart: Intent-Driven Zero-Shot Chart Generation from Documents

Akriti Jain, Pritika Ramu, Aparna Garimella, Apoorv Saxena

机构 * Adobe Research（Adobe研究院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2025 Main Conference

Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs

Aryan Gulati, Brando Miranda, Eric Chen, Emily Xia, Kai Fronsdal, Bruno Dumont, Elyas Obbad, Sanmi Koyejo

机构 * Department of Computer Science, Stanford University, Stanford, USA（计算机科学系，斯坦福大学，斯坦福，美国） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages total (10-page main paper + 17-page appendix), 12 figures, 6 tables. Submitted to ICML 2025 (under review)

Journal ref ICML 2025

Rule Synergy Analysis using LLMs: State of the Art and Implications

Bahar Bateni, Benjamin Pratt, Jim Whitehead

专题命中 评测与基准 ：large language model(abstract,comments)；language model(abstract,comments)；分类 cs.CL

Comments Submitted for publication at the IEEE Transactions on Games 2024, Special Issue on Large Language Models and Games (10 pages excluding appendix, 3 figures)

Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents

Manh Hung Nguyen, Victor-Alexandru Pădurean, Alkis Gotovos, Sebastian Tschiatschek, Adish Singla

机构 * MPI-SWS ； University of Vienna（维也纳大学） 纠错

专题命中 评测与基准 ：LLM(title)；分类 cs.AI

Comments AIED'25 paper

Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts

Jiaqi Deng, Yuho Lee, Nicole Hee-Yeon Kim, Hyangsuk Min, Taewon Yun, Minjeong Ban, Kim Yul, Hwanjun Song

机构 * Korea Advanced Institute of Science and Technology（韩国科学技术院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 (Main)

PediatricsMQA: a Multi-modal Pediatrics Question Answering Benchmark

Adil Bahaj, Oumaima Fadi, Mohamed Chetouani, Mounir Ghogho

机构 * Mohammed 6 Polytechnic University（摩洛哥6号理工学院） ； International University of Rabat（拉巴特国际大学） ； Institut des Systèmes Intelligents et de Robotique（智能系统与机器人研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

mSTEB: Massively Multilingual Evaluation of LLMs on Speech and Text Tasks

Luel Hagos Beyene, Vivek Verma, Min Ma, Jesujoba O. Alabi, Fabian David Schmidt, Joyce Nakatumba-Nabende, David Ifeoluwa Adelani

机构 * AIMS RIC ； NM-AIST ； Mila - Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； Google DeepMind（谷歌DeepMind） ； Saarland University（萨尔兰大学） ； University of Würzburg（维尔茨堡大学） ； Makerere University（Makerere大学） ； McGill University（麦吉尔大学） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ASRU 2025

HEAL: A Hypothesis-Based Preference-Aware Analysis Framework

Yifu Huo, Chenglong Wang, Qiren Zhu, Shunjie Xing, Tong Xiao, Chunliang Zhang, Tongran Liu, Jinbo Zhu

机构 * School of Computer Science and Engineering, Northeastern University（计算机科学与工程学院，东北大学） ； NiuTrans Research（NiuTrans研究院） ； CAS Key Laboratory of Behavioral Science, Institute of Psychology, CAS（行为科学重点实验室，心理学研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2025 Findings

Interestingness First Classifiers

Ryoma Sato

机构 * National Institute of Informatics（国立信息研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 14 pages

Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities

Trisanth Srinivasan, Santosh Patapati

机构 * Cyrion Labs（Cyron实验室） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, Pan Li

机构 * Case Western Reserve University（凯斯西储大学） ； Hangzhou Dianzi University（杭州电子科技大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to EMNLP2025 main conference (poster)

Step-Audio 2 Technical Report

Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, Yuxin Zhang, Zhao You, Brian Li, Changyi Wan, Hanpeng Hu, Jiangjie Zhen, Siyu Chen, Song Yuan, Xuelin Zhang, Yimin Jiang, Yu Zhou, Yuxiang Yang, Bingxin Li, Buyun Ma, Changhe Song, Dongqing Pang, Guoqiang Hu, Haiyang Sun, Kang An, Na Wang, Shuli Gao, Wei Ji, Wen Li, Wen Sun, Xuan Wen, Yong Ren, Yuankai Ma, Yufan Lu, Bin Wang, Bo Li, Changxin Miao, Che Liu, Chen Xu, Dapeng Shi, Dingyuan Hu, Donghang Wu, Enle Liu, Guanzhe Huang, Gulin Yan, Han Zhang, Hao Nie, Haonan Jia, Hongyu Zhou, Jianjian Sun, Jiaoren Wu, Jie Wu, Jie Yang, Jin Yang, Junzhe Lin, Kaixiang Li, Lei Yang, Liying Shi, Li Zhou, Longlong Gu, Ming Li, Mingliang Li, Mingxiao Li, Nan Wu, Qi Han, Qinyuan Tan, Shaoliang Pang, Shengjie Fan, Siqi Liu, Tiancheng Cao, Wanying Lu, Wenqing He, Wuxun Xie, Xu Zhao, Xueqi Li, Yanbo Yu, Yang Yang, Yi Liu, Yifan Lu, Yilei Wang, Yuanhao Ding, Yuanwei Liang, Yuanwei Lu, Yuchu Luo, Yuhe Yin, Yumeng Zhan, Yuxiang Zhang, Zidong Yang, Zixin Zhang, Binxing Jiao, Daxin Jiang, Heung-Yeung Shum, Jiansheng Chen, Jing Li, Xiangyu Zhang, Yibo Zhu

机构 * StepFun Audio Team（StepFun音频团队） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments v3: Added introduction and evaluation results of Step-Audio 2 mini

Towards New Benchmark for AI Alignment & Sentiment Analysis in Socially Important Issues: A Comparative Study of Human and LLMs in the Context of AGI

Ljubisa Bojic, Dylan Seychell, Milan Cabarkapa

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 34 pages, 3 figures