语言大模型 / LLM

Behavioral Fingerprinting of Large Language Models

Zehua Pei, Hui-Ling Zhen, Ying Zhang, Zhiyuan Yang, Xing Li, Xianzhi Yu, Mingxuan Yuan, Bei Yu

机构 * The Chinese University of Hong Kong（香港中文大学） ； Noah’s Ark Lab, Huawei（华为诺亚实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Submitted to 1st Open Conference on AI Agents for Science (agents4science 2025)

OPRA-Vis: Visual Analytics System to Assist Organization-Public Relationship Assessment with Large Language Models

Sangbong Yoo, Seongbum Seo, Chanyoung Yoon, Hyelim Lee, Jeong-Nam Kim, Chansoo Kim, Yun Jang, Takanori Fujiwara

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Large Language Model-Driven Dynamic Assessment of Grammatical Accuracy in English Language Learner Writing

Timur Jaganov, John Blake, Julián Villegas, Nicholas Carr

机构 * National Institute of Standards（国家标准 institute） ； Department of Physics, Colorado State University（物理系，科罗拉多州立大学） ； Electrical Engineering Department, University of Colorado（电气工程系，科罗拉多大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 8 Figures. This work has been submitted to the IEEE for possible publication

Journal ref IEEE ACCESS, 2025, Volume 13, pp. 151538-151550

Memorization $\neq$ Understanding: Do Large Language Models Have the Ability of Scenario Cognition?

Boxiang Ma, Ru Li, Yuanlong Wang, Hongye Tan, Xiaoli Li

机构 * School of Computer and Information Technology, Shanxi University（山西大学计算机与信息学院） ； Information Systems Technology and Design, Singapore University of Technology and Design（新加坡科技设计大学信息系统技术与设计） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP 2025 Main Conference

INSEva: A Comprehensive Chinese Benchmark for Large Language Models in Insurance

Shisong Chen, Qian Zhu, Wenyan Yang, Chengyi Yang, Zhong Wang, Ping Wang, Xuan Lin, Bo Xu, Daqian Li, Chao Yuan, Licai Qi, Wanqing Xu, sun zhenxing, Xin Lu, Shiqiang Xiong, Chao Chen, Haixiang Hu, Yanghua Xiao

机构 * Ant Group（蚂蚁集团） ； Fudan University（复旦大学） ； East China Normal University（东华大学） ； Donghua University（东华大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Under review

Comparative Evaluation of Large Language Models for Test-Skeleton Generation

Subhang Boorlagadda, Nitya Naga Sai Atluri, Muhammet Mustafa Olmez, Edward F. Gehringer

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)

Comments Forthcoming in Frontiers in Education (FIE 2025), Nashville, Tennessee, USA, Nov 2-5, 2025

Do Large Language Models Need Intent? Revisiting Response Generation Strategies for Service Assistant

Inbal Bolshinsky, Shani Kupiec, Almog Sasson, Yehudit Aperstein, Alexander Apartsin

专题命中 评测与基准 ：language model(title,abstract)；large language model(title)；分类 cs.CL、cs.LG

Comments 7 pages, 1 figure

Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment

Jiahuan Pei, Fanghua Ye, Xin Sun, Wentao Deng, Koen Hindriks, Junxiao Wang

机构 * Vrije University of Amsterdam（阿姆斯特丹自由大学） ； University College London（伦敦大学学院） ； University of Amsterdam（阿姆斯特丹大学） ； National Institute of Informatics（日本信息处理学会） ； Shandong University（山东大学） ； Guangzhou University（广州大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 14 pages, accepted by EMNLP 2025

Multi-Modal Vision vs. Text-Based Parsing: Benchmarking LLM Strategies for Invoice Processing

David Berghaus, Armin Berger, Lars Hillebrand, Kostadin Cvejoski, Rafet Sifa

机构 * Fraunhofer IAIS（弗劳恩霍夫人工智能研究所） ； Lamarr Institute（拉马尔研究所） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；prompting(abstract)

MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Design

Zishang Qiu, Xinan Chen, Long Chen, Ruibin Bai

机构 * School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波分校计算机科学学院） ； College of Teacher Education, Zhejiang Normal University（浙江师范大学教师教育学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

First Steps Towards Overhearing LLM Agents: A Case Study With Dungeons & Dragons Gameplay

Andrew Zhu, Evan Osgood, Chris Callison-Burch

机构 * University of Pennsylvania（宾夕法尼亚大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 5 figures. COLM 2025 Workshop on AI Agents

Personality as a Probe for LLM Evaluation: Method Trade-offs and Downstream Effects

Gunmay Handa, Zekun Wu, Adriano Koshiyama, Philip Treleaven

机构 * University College London（伦敦大学学院） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions

Xinbei Ma, Yiting Wang, Yao Yao, Tongxin Yuan, Aston Zhang, Zhuosheng Zhang, Hai Zhao

机构 * School of Computer Science（计算机学院） ； Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering（智能交互与认知工程重点实验室） ； Shanghai Jiao Tong University（上海交通大学） ； Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3（Web3可信数据流通与治理上海市重点实验室） ； GenAI, Meta（Meta GenAI） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL 2025

ThumbnailTruth: A Multi-Modal LLM Approach for Detecting Misleading YouTube Thumbnails Across Diverse Cultural Settings

Wajiha Naveed, Zartash Afzal Uzmi, Zafar Ayyub Qazi

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction

Zhiyuan Zeng, Jiashuo Liu, Siyuan Chen, Tianci He, Yali Liao, Yixiao Tian, Jinpeng Wang, Zaiyuan Wang, Yang Yang, Lingyue Yin, Mingren Yin, Zhenwei Zhu, Tianle Cai, Zehui Chen, Jiecao Chen, Yantao Du, Xiang Gao, Jiacheng Guo, Liang Hu, Jianpeng Jiao, Xiangsheng Li, Jingkai Liu, Shuang Ni, Zhoufutu Wen, Ge Zhang, Kaiyuan Zhang, Xin Zhou, Jose Blanchet, Xipeng Qiu, Mengdi Wang, Wenhao Huang

机构 * Fudan University（复旦大学） ； Stanford University（斯坦福大学） ； Princeton University（普林斯顿大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI、cs.LG

Comments Technical report, 51 pages. Update the results

Masked Diffusion Language Models with Frequency-Informed Training

Despoina Kosmopoulou, Efthymios Georgiou, Vaggelis Dorovatas, Georgios Paraskevopoulos, Alexandros Potamianos

机构 * National Technical University of Athens（国家技术大学雅典） ； Archimedes RU, Athena RC（阿基米德RU，雅典RC） ； University of Bern（伯恩大学） ； Institute of Language and Signal Processing, Athena RC（语言与信号处理研究所，雅典RC） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Comments Preprint

Classification of kinetic-related injury in hospital triage data using NLP

Midhun Shyam, Jim Basilakis, Kieran Luken, Steven Thomas, John Crozier, Paul M. Middleton, X. Rosalind Wang

机构 * School of Computer, Data and Mathematical Sciences（计算机、数据与数学科学学院） ； South Western Emergency Research Institute（西南急诊研究机构） ； Ingham Institute of Applied Medical Research（ingham应用医学研究学院） ； Liverpool Hospital（利物浦医院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted as a short paper for publishing at ADMA 2025 (https://adma2025.github.io), with Supplementary Material available at https://github.com/CRMDS/Kinetic-Injury-Triage

ATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation

Mohammed Khalil, Mohammed Sabry

机构 * ADAPT/DCU（ADAPT/都柏林大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments ArabicNLP 2025

FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases

Matteo Poggi, Fabio Tosi

机构 * Department of Computer Science and Engineering (DISI)（计算机科学与工程系） ； Advanced Research Center on Electronic System (ARCES)（电子系统高级研究中心） ； University of Bologna（博洛尼亚大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)

Comments ICCV 2025 - Project Page: https://flowseek25.github.io/ - Code: https://github.com/mattpoggi/flowseek

RINSER: Accurate API Prediction Using Masked Language Models

Muhammad Ejaz Ahmed, Christopher Cody, Muhammad Ikram, Sean Lamont, Alsharif Abuadbba, Seyit Camtepe, Surya Nepal, Muhammad Ali Kaafar

专题命中 评测与基准 ：language model(title,abstract)

Comments 16 pages, 8 figures

A foundation model for atomistic materials chemistry

Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M. Elena, Dávid P. Kovács, Janosh Riebesell, Xavier R. Advincula, Mark Asta, Matthew Avaylon, William J. Baldwin, Fabian Berger, Noam Bernstein, Arghya Bhowmik, Filippo Bigi, Samuel M. Blau, Vlad Cărare, Michele Ceriotti, Sanggyu Chong, James P. Darby, Sandip De, Flaviano Della Pia, Volker L. Deringer, Rokas Elijošius, Zakariya El-Machachi, Fabio Falcioni, Edvin Fako, Andrea C. Ferrari, John L. A. Gardner, Mikolaj J. Gawkowski, Annalena Genreith-Schriever, Janine George, Rhys E. A. Goodall, Jonas Grandel, Clare P. Grey, Petr Grigorev, Shuang Han, Will Handley, Hendrik H. Heenen, Kersti Hermansson, Christian Holm, Cheuk Hin Ho, Stephan Hofmann, Jad Jaafar, Konstantin S. Jakob, Hyunwook Jung, Venkat Kapil, Aaron D. Kaplan, Nima Karimitari, James R. Kermode, Panagiotis Kourtis, Namu Kroupa, Jolla Kullgren, Matthew C. Kuner, Domantas Kuryla, Guoda Liepuoniute, Chen Lin, Johannes T. Margraf, Ioan-Bogdan Magdău, Angelos Michaelides, J. Harry Moore, Aakash A. Naik, Samuel P. Niblett, Sam Walton Norwood, Niamh O'Neill, Christoph Ortner, Kristin A. Persson, Karsten Reuter, Andrew S. Rosen, Louise A. M. Rosset, Lars L. Schaaf, Christoph Schran, Benjamin X. Shi, Eric Sivonxay, Tamás K. Stenczel, Viktor Svahn, Christopher Sutton, Thomas D. Swinburne, Jules Tilly, Cas van der Oord, Santiago Vargas, Eszter Varga-Umbrich, Tejs Vegge, Martin Vondrák, Yangshuai Wang, William C. Witt, Thomas Wolf, Fabian Zills, Gábor Csányi

专题命中 评测与基准 ：foundation model(title,abstract)

Comments 154 pages, 83 figures, 80MB PDF

Graph RAG as Human Choice Model: Building a Data-Driven Mobility Agent with Preference Chain

Kai Hu, Parfait Atchade-Adelomou, Carlo Adornetto, Adrian Mora-Carrero, Luis Alonso-Pastor, Ariel Noyman, Yubo Liu, Kent Larson

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Assessing the Sensitivity and Alignment of FOL Closeness Metrics

Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi

机构 * Department of Data Science & AI, Monash University（数据科学与人工智能系，莫纳什大学） ； College of Engineering and Computer Science, VinUniversity（工程与计算机科学学院，文大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025

From Post To Personality: Harnessing LLMs for MBTI Prediction in Social Media

Tian Ma, Kaiyu Feng, Yu Rong, Kangfei Zhao

机构 * Beijing Institute of Technology（北京理工大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Journal ref CIKM 2025 Short Paper (Technical Report)

DecMetrics: Structured Claim Decomposition Scoring for Factually Consistent LLM Outputs

Minghui Huang

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 评测与基准 ：LLM(title)；分类 cs.CL、cs.AI

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks

Joshua H. Davis, Daniel Nichols, Ishan Khillan, Abhinav Bhatele

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 5 figures

Social Bias in Multilingual Language Models: A Survey

Lance Calvin Lim Gamboa, Yue Feng, Mark Lee

机构 * School of Computer Science, University of Birmingham（伯明翰大学计算机科学学院） ； Department of Information Systems and Computer Science, Ateneo de Manila University（马尼拉大学信息系统与计算机科学系） 纠错

专题命中 评测与基准 ：language model(title)；分类 cs.CL

Comments Accepted into EMNLP 2025 Main Conference

SparkUI-Parser: Enhancing GUI Perception with Robust Grounding and Parsing

Hongyi Jing, Jiafu Chen, Chen Rao, Ziqiang Dang, Jiajie Teng, Tianyi Chu, Juncheng Mo, Shuo Fang, Huaizhong Lin, Rui Lv, Chenguang Ma, Lei Zhao

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection

Yihan Chen, Jiawei Chen, Guozhao Mo, Xuanang Chen, Ben He, Xianpei Han, Le Sun

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training

Figarri Keisha, Zekun Wu, Ze Wang, Adriano Koshiyama, Philip Treleaven

机构 * Holistic AI ； University College London（伦敦大学学院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL