语言大模型 / LLM

Design and Application of Multimodal Large Language Model Based System for End to End Automation of Accident Dataset Generation

MD Thamed Bin Zaman Chowdhury, Moazzem Hossain

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments This paper is accepted for presentation in TRB annual meeting 2026. The version presented here is the preprint version before peer review process

Adapting Large Language Models for Character-based Augmentative and Alternative Communication

Dylan Gaines, Keith Vertanen

机构 * Dylan Gaines Department of Computer Science, Michigan Technological University, Houghton, MI, USA（达西·甘斯 计算机科学系，密歇根技术大学，霍顿，MI，美国） ； Keith Vertanen（凯斯·维塔内） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments To appear in Findings of EMNLP 2025

Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretations

Ricardo Gonzalez Penuela, Felipe Arias-Russi, Victor Capriles

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 7 pages, 2 figure, 2 tables, CV4A11y Workshop at ICCV 2025

Emergent evaluation hubs in a decentralizing large language model ecosystem

Manuel Cebrian, Tomomi Kito, Raul Castro Fernandez

机构 * Center for Automation and Robotics, Spanish National Research Council, Spain（自动化与机器人中心，西班牙国家研究理事会） ； Graduate School of Creative Science and Engineering, Waseda University, Japan（创意科学与工程研究生院，早稻田大学） ； Department of Computer Science, University of Chicago, USA（计算机科学系，芝加哥大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 15 pages, 11 figures, 3 tables

Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports

Punit Kumar Singh, Nishant Kumar, Akash Ghosh, Kunal Pasad, Khushi Soni, Manisha Jaishwal, Sriparna Saha, Syukron Abu Ishaq Alfarozi, Asres Temam Abagissa, Kitsuchart Pasupa, Haiqin Yang, Jose G Moreno

机构 * Indian Institute of Technology Patna（印度理工学院帕纳布分校） ； Sardar Patel Institute of Technology（萨达尔·帕特尔技术学院） ； Universitas Gadjah Mada（加查马大学） ； King Mongkut’s Institute of Technology Ladkrabang（拉差班国王技术学院） ； Shenzhen Technology University（深圳技术大学） ； Université de Toulouse（图卢兹大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；small language model(abstract)；prompting(abstract)

Comments 52 pages, 56 figures; appearing at EMNLP'25

Deriving Strategic Market Insights with Large Language Models: A Benchmark for Forward Counterfactual Generation

Keane Ong, Rui Mao, Deeksha Varshney, Paul Pu Liang, Erik Cambria, Gianmarco Mengaldo

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Published at Empirical Methods in Natural Language Processing 2025 (Main Conference) (Oral)

The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation

Zarreen Reza

机构 * Independent researcher（独立研究者） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

POLAR: Automating Cyber Threat Prioritization through LLM-Powered Assessment

Luoxi Tang, Yuqiao Meng, Ankita Patra, Weicheng Ma, Muchao Ye, Zhaohan Xi

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 25 pages

Longitudinal Monitoring of LLM Content Moderation of Social Issues

Yunlang Dai, Emma Lurie, Danaé Metaxa, Sorelle A. Friedler

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

How can AI agents support journalists' work? An experiment with designing an LLM-driven intelligent reporting system

Vasileios Maltezos, Roman Kyrychenko, Aleksi Knuutila

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 8 pages, 1 table

Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks

Dongjun Kim, Gyuho Shim, Yongchan Chun, Minhyuk Kim, Chanjun Park, Heuiseok Lim

机构 * Department of Computer Science and Engineering, Korea University（韩国大学计算机科学与工程系） ； School of Software, Soongsil University（顺天大学软件学院） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 5 figures. Accepted to EMNLP 2025 main conference

Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style LLM Evaluation

Raphael Tang, Crystina Zhang, Wenyan Li, Carmen Lai, Pontus Stenetorp, Yao Lu

机构 * Centre for Artificial Intelligence, University College London（人工智能中心，伦敦大学学院） ； University of Waterloo（滑铁卢大学） ； University of Copenhagen（哥本哈根大学） ； Research and Development Center for Large Language Models, National Institute of Informatics（大语言模型研究与发展中心，日本信息处理学会） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 6 pages, 4 figures

Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking

Liyan Xie, Muhammad Siddeek, Mohamed Seif, Andrea J. Goldsmith, Mengdi Wang

机构 * Department of Industrial and Systems Engineering, University of Minnesota（工业与系统工程系，明尼苏达大学） ； Google（谷歌） ； Department of Electrical and Computer Engineering, Princeton University（电气工程与计算机科学系，普林斯顿大学） ； Stony Brook University（石溪大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；language model(abstract)；分类 cs.LG

ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models

Krishna Teja Chitty-Venkata, Murali Emani

机构 * Argonne National Laboratory（阿贡国家实验室） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.LG

Comments Preprint

Evaluating New AI Cell Foundation Models on Challenging Kidney Pathology Cases Unaddressed by Previous Foundation Models

Runchen Wang, Junlin Guo, Siqi Lu, Ruining Deng, Zhengyi Lu, Yanfan Zhu, Yuechen Yang, Chongyu Qu, Yu Wang, Shilin Zhao, Catie Chang, Mitchell Wilkes, Mengmeng Yin, Haichun Yang, Yuankai Huo

机构 * Vanderbilt University（范德比尔特大学） ； Weill Cornell Medicine（韦尔医学院） ； Vanderbilt University Medical Center（范德比尔特大学医学中心） ； Department of Electrical and Computer Engineering（电气与计算机工程系） ； Department of Radiology（放射科） ； Department of Computer Science（计算机科学系） ； Department of Biostatistics（生物统计学系） ； Department of Pathology, Microbiology and Immunology（病理学、微生物学和免疫学系） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.AI

Pre-Hoc Predictions in AutoML: Leveraging LLMs to Enhance Model Selection and Benchmarking for Tabular datasets

Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo, Sachin Sharma, John D. Kelleher

机构 * IBM Research Europe（IBM欧洲研究院） ； Trinity College Dublin（都柏林三一学院） ； Technological University Dublin（都柏林技术大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Oral Presentations ADAPT Annual Scientific Conference 2025

Context Matters: Comparison of commercial large language tools in veterinary medicine

Tyler J Poore, Christopher J Pinard, Aleena Shabbir, Andrew Lagree, Andre Telfer, Kuan-Chuen Wu

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 4 Figures, 10 pages

Mamba Outpaces Reformer in Stock Prediction with Sentiments from Top Ten LLMs

Lokesh Antony Kadiyala, Amir Mirzaeinia

机构 * University of North Texas USA（美国北德克萨斯大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

GeoSQL-Eval: First Evaluation of LLMs on PostGIS-Based NL2GeoSQL Queries

Shuyang Hou, Haoyue Jiao, Ziqi Liu, Lutong Xie, Guanyu Chen, Shaowen Wu, Xuefeng Guan, Huayi Wu

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Evaluating LLMs for Combinatorial Optimization: One-Phase and Two-Phase Heuristics for 2D Bin-Packing

Syed Mahbubul Huq, Daniel Brito, Daniel Sikar, Chris Child, Tillman Weyde, Rajesh Mojumder

机构 * Department of Computer Science, School of Science and Technology（计算机科学系，科学与技术学院） ； University of London（伦敦大学） ； BRAC University（BRAC大学） 纠错

专题命中 评测与基准 ：LLM(abstract,comments)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 1 table, 6 figures. 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Accepted for the Workshop: Evaluating the Evolving LLM Lifecycle Benchmarks, Emergent Abilities, and Scaling

IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol

Ningyuan Yang, Guanliang Lyu, Mingchen Ma, Yiyi Lu, Yiming Li, Zhihui Gao, Hancheng Ye, Jianyi Zhang, Tingjun Chen, Yiran Chen

机构 * Department of Electrical and Computer Engineering, Duke University（电子与计算机工程系，杜克大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Generating Difficult-to-Translate Texts

Vilém Zouhar, Wenda Xu, Parker Riley, Juraj Juraska, Mara Finkelstein, Markus Freitag, Daniel Deutsch

机构 * Google（谷歌） ； ETH Zurich（苏黎世联邦理工学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

A Flexible Method for Behaviorally Measuring Alignment Between Human and Artificial Intelligence Using Representational Similarity Analysis

Mattson Ogg, Ritwik Bose, Jamie Scharf, Christopher Ratto, Michael Wolmetz

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMs

Jiyao Liu, Jinjie Wei, Wanying Qu, Chenglong Ma, Junzhi Ning, Yunheng Li, Ying Chen, Xinzhe Luo, Pengcheng Chen, Xin Gao, Ming Hu, Huihui Xu, Xin Wang, Shujian Gao, Dingkang Yang, Zhongying Deng, Jin Ye, Lihao Liu, Junjun He, Ningsheng Xu

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Imperial College London（帝国理工学院） ； University of Cambridge（剑桥大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 26 pages, 13 figures

BioinfoMCP: A Unified Platform Enabling MCP Interfaces in Agentic Bioinformatics

Florensia Widjaja, Zhangtianyi Chen, Juexiao Zhou

机构 * School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK Shenzhen)（数据科学学院，香港中文大学（深圳）） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 8 figures, 3 tables

Predictive Modeling and Explainable AI for Veterinary Safety Profiles, Residue Assessment, and Health Outcomes Using Real-World Data and Physicochemical Properties

Hossein Sholehrasa, Xuan Xu, Doina Caragea, Jim E. Riviere, Majid Jaberi-Douraki

机构 * DATA Consortium and FARAD Program, Kansas State University, Olathe, KS, USA（1DATA Consortium和FARAD计划，堪萨斯州立大学） ； Department of Computer Science, Kansas State University, Manhattan, KS, USA（计算机科学系，堪萨斯州立大学） ； Department of Statistics, Kansas State University, Manhattan, KS, USA（统计学系，堪萨斯州立大学） ； Department of Mathematics, Kansas State University, Olathe, KS, USA（数学系，堪萨斯州立大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

A-VERT: Agnostic Verification with Embedding Ranking Targets

Nicolás Aguirre, Ramiro Caso, Ramiro Rodríguez Colmeiro, Mauro Santelli, Joaquín Toranzo Calderón

机构 * GIAR, UTN-FRBA Pnyx AI（GIAR、UTN-FRBA 人工智能实验室） ； IIF–SADAF–CONICET Universidad de Buenos Aires（IIF–SADAF–CONICET 乌拉圭大学） ； UTN-FRBA Pnyx AI（UTN-FRBA 人工智能实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 19 pages, 7 figures, code available at https://github.com/pnyxai/a-vert, authors in alphabetical order

CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation

Anirudh Khatry, Robert Zhang, Jia Pan, Ziteng Wang, Qiaochu Chen, Greg Durrett, Isil Dillig

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） ； New York University（纽约大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments To be published at COLM, 2025

Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage

Siddhant Arora, Haidar Khan, Kai Sun, Xin Luna Dong, Sajal Choudhary, Seungwhan Moon, Xinyuan Zhang, Adithya Sagar, Surya Teja Appini, Kaushik Patnaik, Sanat Sharma, Shinji Watanabe, Anuj Kumar, Ahmed Aly, Yue Liu, Florian Metze, Zhaojiang Lin

机构 * Meta ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；post-training(abstract)；分类 cs.CL

CardioRAG: A Retrieval-Augmented Generation Framework for Multimodal Chagas Disease Detection

Zhengyang Shen, Xuehao Zhai, Hua Tu, Mayue Shi

机构 * Department of Electrical and Electronic Engineering Imperial College London（帝国理工学院电子与电气工程系） ； Department of Civil and Environmental Engineering Imperial College London（帝国理工学院土木与环境工程系） ； Institute of Biomedical Engineering Department of Engineering Science University of Oxford（牛津大学生物医学工程研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 4 pages, 2 figures. Accepted for oral presentation at the 52nd international Computing in Cardiology Conference (CinC2025)