语言大模型 / LLM

MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers

Ziyang Luo, Zhiqi Shen, Wenzhuo Yang, Zirui Zhao, Prathyusha Jwalapuram, Amrita Saha, Doyen Sahoo, Silvio Savarese, Caiming Xiong, Junnan Li

机构 * Salesforce AI Research（Salesforce人工智能研究） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Website: https://mcp-universe.github.io

The NordDRG AI Benchmark for Large Language Models

Tapio Pitkäranta

机构 * Department of Computer Science and Engineering（计算机科学与工程系） ； Aalto University（阿alto大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 23 pages, 4 figures

Confidence Estimation for Text-to-SQL in Large Language Models

Sepideh Entezari Maleki, Mohammadreza Pourreza, Davood Rafiei

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

The Man Behind the Sound: Demystifying Audio Private Attribute Profiling via Multimodal Large Language Model Agents

Lixu Wang, Kaixiang Yao, Xinfeng Li, Dong Yang, Haoyang Li, Xiaofeng Wang, Wei Dong

机构 * Nanyang Technological University（南洋理工大学） ； The University of Tokyo（东京大学） ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 22 pages, 4 figures

FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics

David Park, Shuhang Li, Yi Huang, Xihaier Luo, Haiwang Yu, Yeonju Go, Christopher Pinkenburg, Yuewei Lin, Shinjae Yoo, Joseph Osborn, Jin Huang, Yihui Ren

专题命中 评测与基准 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness

Scott Blyth, Sherlock A. Licorish, Christoph Treude, Markus Wagner

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；prompting(abstract)

Towards LLM-generated explanations for Component-based Knowledge Graph Question Answering Systems

Dennis Schiese, Aleksandr Perevalov, Andreas Both

专题命中 评测与基准 ：LLM(title,comments)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Presented at ICWI 2024, Zagreb. Released with ISBN: 978-989-8704-62-7. Data source: https://figshare.com/articles/dataset/Towards_LLM-generated_explanations_for_component-based_knowledge_graph_question_answering_systems/27079687

Measuring LLM Code Generation Stability via Structural Entropy

Yewei Song, Tiezhu Sun, Xunzhu Tang, Prateek Rajput, Tegawende F. Bissyande, Jacques Klein

机构 * The Interdisciplinary Centre for Security, Reliability and Trust University of Luxembourg（安全、可靠与信任跨学科研究中心卢森堡大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ASE-NIER

Pixels to Play: A Foundation Model for 3D Gameplay

Yuguang Yue, Chris Green, Samuel Hunt, Irakli Salia, Wenzhe Shi, Jonathan J Hunt

机构 * Player2 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Journal ref Conference on Games 2025 (Short paper)

ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine

Junying Chen, Zhenyang Cai, Zhiheng Liu, Yunjin Yang, Rongsheng Wang, Qingying Xiao, Xiangyi Feng, Zhan Su, Jing Guo, Xiang Wan, Guangjun Yu, Haizhou Li, Benyou Wang

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

DeepTelecom: A Digital-Twin Deep Learning Dataset for Channel and MIMO Applications

Bohao Wang, Zehua Jiang, Zhenyu Yang, Chongwen Huang, Yongliang Shen, Siming Jiang, Chen Zhu, Zhaohui Yang, Richeng Jin, Zhaoyang Zhang, Sami Muhaidat, Merouane Debbah

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Benchmarking Pre-Trained Time Series Models for Electricity Price Forecasting

Timothée Hornek Amir Sartipi, Igor Tchappi, Gilbert Fridgen

机构 * SnT - Interdisciplinary Center for Security, Reliability and Trust（SnT跨学科安全、可靠与信任中心） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Enhancing Depression-Diagnosis-Oriented Chat with Psychological State Tracking

Yiyang Gu, Yougen Zhou, Qin Chen, Ningning Zhou, Jie Zhou, Aimin Zhou, Liang He

机构 * School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院） ； Shanghai Institute of Al for Education, East China Normal University（东华大学上海人工智能教育研究所） ； School of Psychology and Cognitive Science, East China Normal University（东华大学心理学与认知科学学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NLPCC 2025

Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motion

Mona Sheikh Zeinoddin, Mobarak I. Hoque, Zafer Tandogdu, Greg Shaw, Matthew J. Clarkson, Evangelos Mazomenos, Danail Stoyanov

机构 * UCL Hawkes Institute, University College London, UK \ mona.zeinoddin.22 ； Interventional Science, University College London, UK 纠错

专题命中 评测与基准 ：foundation model(title,abstract)

Bench2ADVLM: A Closed-Loop Benchmark for Vision-language Models in Autonomous Driving

Tianyuan Zhang, Ting Jin, Lu Wang, Jiangfan Liu, Siyuan Liang, Mingchuan Zhang, Aishan Liu, Xianglong Liu

机构 * Beihang University（北航大学） ； Nanyang Technological University（南洋理工大学） ； Henan University of Science and Technology（河南科技大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)

Assessing the Quality and Security of AI-Generated Code: A Quantitative Analysis

Abbas Sabra, Olivier Schmitt, Joseph Tyler

机构 * Sonar 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

PAPPL: Personalized AI-Powered Progressive Learning Platform

Shayan Bafandkar, Sungyong Chung, Homa Khosravian, Alireza Talebpour

机构 * The Grainger College of Engineering, Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, 205 N Matthews, Urbana, IL 61801（伊利诺伊大学厄巴纳-香槟分校格拉inger工程学院，土木与环境工程系） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples

Haiquan Hu, Jiazhi Jiang, Shiyou Xu, Ruhan Zeng, Tian Wang

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Submit to AAAI 2026

Benchmarking Vector, Graph and Hybrid Retrieval Augmented Generation (RAG) Pipelines for Open Radio Access Networks (ORAN)

Sarat Ahmad, Zeinab Nezami, Maryam Hafeez, Syed Ali Raza Zaidi

机构 * School of Electronic and Electrical Engineering（电子与电气工程学院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluation

Yutong Bian, Xianhao Lin, Yupeng Xie, Tianyang Liu, Mingchen Zhuge, Siyuan Lu, Haoming Tang, Jinlin Wang, Jiayi Zhang, Jiaqi Chen, Xiangru Tang, Yongxin Ni, Sirui Hong, Chenglin Wu

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge

Guanghao Jin, Jingpei Wu, Tianpei Guo, Yiyi Niu, Weidong Zhou, Guoyang Liu

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG

JRDB-Reasoning: A Difficulty-Graded Benchmark for Visual Reasoning in Robotics

Simindokht Jahangard, Mehrzad Mohammadi, Yi Shen, Zhixi Cai, Hamid Rezatofighi

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Reinforcement Learning to Rank Using Coarse-grained Rewards

Yiteng Tu, Zhichao Xu, Tao Yang, Weihang Su, Yujia Zhou, Yiqun Liu, Fen Lin, Qin Liu, Qingyao Ai

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

MetaWild: A Multimodal Dataset for Animal Re-Identification with Environmental Metadata

Yuzhuo Li, Di Zhao, Tingrui Qiao, Yihao Wu, Bo Pang, Yun Sing Koh

机构 * University of Auckland（奥克兰大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.LG

Comments 7 pages, 6 figures

Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach

Kezia Oketch, John P. Lalor, Ahmed Abbasi

机构 * Department of IT, Analytics, and Operations University of Notre Dame（信息科技、分析与运营系 纽约大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL

From Image Captioning to Visual Storytelling

Admitos Passadakis, Yingjin Song, Albert Gatt

机构 * TUDelft（代尔夫特理工大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL

Comments 16 pages (including references), 5 figures and 6 tables

Investigating Transcription Normalization in the Faetar ASR Benchmark

Leo Peckham, Michael Ong, Naomi Nagy, Ewan Dunbar

机构 * Department of Linguistics（语言学系） ； Department of Computer Science（计算机科学系） ； Department of French（法语系） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL