语言大模型 / LLM

MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models

Zhongpu Chen, Yinfeng Liu, Long Shi, Xingyan Chen, Yu Zhao, Fuji Ren

机构 * Southwestern University of Finance and Economics（西南财经大学） ； University of Electronic Science and Technology of China（电子科技大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments WWW 2025

ArgCMV: An Argument Summarization Benchmark for the LLM-era

Omkar Gurjar, Agam Goyal, Eshwar Chandrasekharan

机构 * Siebel School of Computing and Data Science（计算与数据科学学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals

Qianli Wang, Van Bach Nguyen, Nils Feldhus, Luis Felipe Villa-Arenas, Christin Seifert, Sebastian Möller, Vera Schmitt

机构 * Quality and Usability Lab, Technische Universität Berlin（柏林技术大学质量与可用性实验室） ； University of Marburg（马尔堡大学） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Deutsche Telekom（德国电信） ； BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at INLG 2025, camera-ready version

Secure Multi-LLM Agentic AI and Agentification for Edge General Intelligence by Zero-Trust: A Survey

Yinqiu Liu, Ruichen Zhang, Haoxiang Luo, Yijing Lin, Geng Sun, Dusit Niyato, Hongyang Du, Zehui Xiong, Yonggang Wen, Abbas Jamalipour, Dong In Kim, Ping Zhang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 35 pages

Tracking World States with Language Models: State-Based Evaluation Using Chess

Romain Harang, Jason Naradowsky, Yaswitha Gujju, Yusuke Miyao

机构 * University of Tokyo（东京大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments Spotlight presentation at ICML 2025 Workshop on Assessing World Models

DemoBias: An Empirical Study to Trace Demographic Biases in Vision Foundation Models

Abu Sufian, Anirudha Ghosh, Debaditya Barman, Marco Leo, Cosimo Distante

机构 * CNR-ISASI（意大利那不勒斯） ； Visva-Bharati（维斯瓦-巴哈蒂） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；language model(abstract)；分类 cs.AI

Comments 6 pages, 4 figures, 13th International Workshop on Biometrics and Forensics (IWBF)

NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision

Xiang Li, Wenyue Hua, Kaijie Zhu, Lingyao Li, Haoyang Ling, Jinkui Chi, Qi Dou, Jindong Wang, Yongfeng Zhang, Xin Ma, Lizhou Fan

机构 * Shandong University, School of Control Science ； Rutgers University, Department of Computer Science 110 Frelinghuysen Road Piscataway New Brunswick NJ USA 08854 ； University of California, Santa Barbara 1210 Cheadle Hall Santa Barbara California USA 93106 ； University of Michigan, School of Information 500 S State St Ann Arbor Michigan USA 48109 ； The Chinese University of Hong Kong, Department of Computer Science ； The College of William \& Mary, School of Computing, Data Sciences \& Physics 200 Stadium Dr Williamsburg Virginia USA 23185 ； The Chinese University of Hong Kong, Department of Psychiatry 9 Chuen On Rd Tai Po New Territories Hong Kong SAR, China 999077 ； Rutgers University, Department of Computer Science ； University of California, Santa Barbara ； University of Michigan, School of Information ； The College of William \& Mary, School of Computing, Data Sciences \& Physics ； The Chinese University of Hong Kong, Department of Psychiatry 纠错

专题命中 评测与基准 ：language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 25 pages, 9 figures, 2 tables

GENIE-ASI: Generative Instruction and Executable Code for Analog Subcircuit Identification

Phuoc Pham, Arun Venkitaraman, Chia-Yu Hsieh, Andrea Bonetti, Stefan Uhlich, Markus Leibl, Simon Hofmann, Eisaku Ohbuchi, Lorenzo Servadei, Ulf Schlichtmann, Robert Wille

机构 * Chair for Design Automation, Technical University of Munich（自动化设计研究所，慕尼黑技术大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Reliable Weak-to-Strong Monitoring of LLM Agents

Neil Kale, Chen Bo Calvin Zhang, Kevin Zhu, Ankit Aich, Paula Rodriguez, Scale Red Team, Christina Q. Knight, Zifan Wang

机构 * Scale AI ； Carnegie Mellon University（卡内基梅隆大学） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 15 figures

Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement

Mohammed Rakibul Hasan, Rafi Majid, Ahanaf Tahmid

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models

Oliver Grainge, Sania Waheed, Jack Stilgoe, Michael Milford, Shoaib Ehsan

专题命中 评测与基准 ：language model(title,abstract)

Comments Accepted to AAAI Fall Symposium 2025 on AI Trustworthiness and Risk Assessment for Challenging Contexts (ATRACC)

Doc2Chart: Intent-Driven Zero-Shot Chart Generation from Documents

Akriti Jain, Pritika Ramu, Aparna Garimella, Apoorv Saxena

机构 * Adobe Research（Adobe研究院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2025 Main Conference

Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs

Aryan Gulati, Brando Miranda, Eric Chen, Emily Xia, Kai Fronsdal, Bruno Dumont, Elyas Obbad, Sanmi Koyejo

机构 * Department of Computer Science, Stanford University, Stanford, USA（计算机科学系，斯坦福大学，斯坦福，美国） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages total (10-page main paper + 17-page appendix), 12 figures, 6 tables. Submitted to ICML 2025 (under review)

Journal ref ICML 2025

Rule Synergy Analysis using LLMs: State of the Art and Implications

Bahar Bateni, Benjamin Pratt, Jim Whitehead

专题命中 评测与基准 ：large language model(abstract,comments)；language model(abstract,comments)；分类 cs.CL

Comments Submitted for publication at the IEEE Transactions on Games 2024, Special Issue on Large Language Models and Games (10 pages excluding appendix, 3 figures)

Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents

Manh Hung Nguyen, Victor-Alexandru Pădurean, Alkis Gotovos, Sebastian Tschiatschek, Adish Singla

机构 * MPI-SWS ； University of Vienna（维也纳大学） 纠错

专题命中 评测与基准 ：LLM(title)；分类 cs.AI

Comments AIED'25 paper

Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts

Jiaqi Deng, Yuho Lee, Nicole Hee-Yeon Kim, Hyangsuk Min, Taewon Yun, Minjeong Ban, Kim Yul, Hwanjun Song

机构 * Korea Advanced Institute of Science and Technology（韩国科学技术院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 (Main)

PediatricsMQA: a Multi-modal Pediatrics Question Answering Benchmark

Adil Bahaj, Oumaima Fadi, Mohamed Chetouani, Mounir Ghogho

机构 * Mohammed 6 Polytechnic University（摩洛哥6号理工学院） ； International University of Rabat（拉巴特国际大学） ； Institut des Systèmes Intelligents et de Robotique（智能系统与机器人研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

mSTEB: Massively Multilingual Evaluation of LLMs on Speech and Text Tasks

Luel Hagos Beyene, Vivek Verma, Min Ma, Jesujoba O. Alabi, Fabian David Schmidt, Joyce Nakatumba-Nabende, David Ifeoluwa Adelani

机构 * AIMS RIC ； NM-AIST ； Mila - Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； Google DeepMind（谷歌DeepMind） ； Saarland University（萨尔兰大学） ； University of Würzburg（维尔茨堡大学） ； Makerere University（Makerere大学） ； McGill University（麦吉尔大学） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ASRU 2025

HEAL: A Hypothesis-Based Preference-Aware Analysis Framework

Yifu Huo, Chenglong Wang, Qiren Zhu, Shunjie Xing, Tong Xiao, Chunliang Zhang, Tongran Liu, Jinbo Zhu

机构 * School of Computer Science and Engineering, Northeastern University（计算机科学与工程学院，东北大学） ； NiuTrans Research（NiuTrans研究院） ； CAS Key Laboratory of Behavioral Science, Institute of Psychology, CAS（行为科学重点实验室，心理学研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2025 Findings

Interestingness First Classifiers

Ryoma Sato

机构 * National Institute of Informatics（国立信息研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 14 pages

Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities

Trisanth Srinivasan, Santosh Patapati

机构 * Cyrion Labs（Cyron实验室） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, Pan Li

机构 * Case Western Reserve University（凯斯西储大学） ； Hangzhou Dianzi University（杭州电子科技大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to EMNLP2025 main conference (poster)

Step-Audio 2 Technical Report

Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, Yuxin Zhang, Zhao You, Brian Li, Changyi Wan, Hanpeng Hu, Jiangjie Zhen, Siyu Chen, Song Yuan, Xuelin Zhang, Yimin Jiang, Yu Zhou, Yuxiang Yang, Bingxin Li, Buyun Ma, Changhe Song, Dongqing Pang, Guoqiang Hu, Haiyang Sun, Kang An, Na Wang, Shuli Gao, Wei Ji, Wen Li, Wen Sun, Xuan Wen, Yong Ren, Yuankai Ma, Yufan Lu, Bin Wang, Bo Li, Changxin Miao, Che Liu, Chen Xu, Dapeng Shi, Dingyuan Hu, Donghang Wu, Enle Liu, Guanzhe Huang, Gulin Yan, Han Zhang, Hao Nie, Haonan Jia, Hongyu Zhou, Jianjian Sun, Jiaoren Wu, Jie Wu, Jie Yang, Jin Yang, Junzhe Lin, Kaixiang Li, Lei Yang, Liying Shi, Li Zhou, Longlong Gu, Ming Li, Mingliang Li, Mingxiao Li, Nan Wu, Qi Han, Qinyuan Tan, Shaoliang Pang, Shengjie Fan, Siqi Liu, Tiancheng Cao, Wanying Lu, Wenqing He, Wuxun Xie, Xu Zhao, Xueqi Li, Yanbo Yu, Yang Yang, Yi Liu, Yifan Lu, Yilei Wang, Yuanhao Ding, Yuanwei Liang, Yuanwei Lu, Yuchu Luo, Yuhe Yin, Yumeng Zhan, Yuxiang Zhang, Zidong Yang, Zixin Zhang, Binxing Jiao, Daxin Jiang, Heung-Yeung Shum, Jiansheng Chen, Jing Li, Xiangyu Zhang, Yibo Zhu

机构 * StepFun Audio Team（StepFun音频团队） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments v3: Added introduction and evaluation results of Step-Audio 2 mini

Towards New Benchmark for AI Alignment & Sentiment Analysis in Socially Important Issues: A Comparative Study of Human and LLMs in the Context of AGI

Ljubisa Bojic, Dylan Seychell, Milan Cabarkapa

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 34 pages, 3 figures

RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits

Kshitij Fadnis, Sara Rosenthal, Maeda Hanafi, Yannis Katsis, Marina Danilevsky

机构 * IBM Research（IBM研究院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

The Return of Structural Handwritten Mathematical Expression Recognition

Jakob Seitz, Tobias Lengfeld, Radu Timofte

机构 * Computer Vision Lab, CAIDAS \& IFI, University of W\"urzburg, W\"urzburg, Germany 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

PersoNo: Personalised Notification Urgency Classifier in Mixed Reality

Jingyao Zheng, Haodi Weng, Xian Wang, Chengbin Cui, Sven Mayer, Chi-lok Tai, Lik-Hang Lee

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments Accepted by ISMAR 2025

Database Entity Recognition with Data Augmentation and Deep Learning

Zikun Fu, Chen Yang, Kourosh Davoudi, Ken Q. Pu

机构 * Faculty of Science Ontario Tech University（科学学院， Ontario Tech 大学） ； College of Engineering Northeastern University（工程学院， Northeastern 大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 6 pages, 5 figures. Accepted at IEEE 26th International Conference on Information Reuse and Integration for Data Science (IRI 2025), San Jose, California, August 6-8, 2025

AI for Statutory Simplification: A Comprehensive State Legal Corpus and Labor Benchmark

Emaan Hariri, Daniel E. Ho

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments 10 pages, 3 figures. To appear in ICAIL 2025

A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics

Rushin H. Gindra, Giovanni Palla, Mathias Nguyen, Sophia J. Wagner, Manuel Tran, Fabian J Theis, Dieter Saur, Lorin Crawford, Tingying Peng

机构 * Helmholtz Munich, Germany（海德堡-慕尼黑赫尔姆霍尔茨研究中心） ； Technical University Munich, Germany（慕尼黑技术大学） ； Microsoft Research, USA（微软研究院） 纠错

专题命中 评测与基准 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments The code is accessible at: https://github.com/peng-lab/hescape