语言大模型 / LLM

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input

Chenxu Li, Zhicai Wang, Yuan Sheng, Xingyu Zhu, Yanbin Hao, Xiang Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 23 pages

MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image

Shezheng Song, Chengxiang He, Shan Zhao, Chengyu Wang, Qian Wan, Tianwei Yan, Meng Wang

机构 * College of Computer, National University of Defense Technology（国防科技大学计算机学院） ； School of Computer and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院） ； CCNU（中国地质大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Design, Results and Industry Implications of the World's First Insurance Large Language Model Evaluation Benchmark

Hua Zhou, Bing Ma, Yufei Zhang, Yi Zhao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 16 pages, 11 models,1 set of evaluation framework,5 core dimensions, 54 sub-indicators, 14,430 high-quality questions

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models

Jingxuan Xu, Ken Deng, Weihao Li, Songwei Yu, Huaixi Tang, Haoyang Huang, Zhiyi Lai, Zizheng Zhan, Yanan Wu, Chenchen Zhang, Kepeng Lei, Yifan Yao, Xinping Lei, Wenqiang Zhu, Zongxian Feng, Han Li, Junqi Xiong, Dailin Li, Zuchen Gao, Kun Wu, Wen Xiang, Ziqi Zhan, Yuanxing Zhang, Wuxuan Gong, Ziyuan Gao, Guanxiang Wang, Yirong Xue, Mengtong Li, Mengfei Xie, Xiaojiang Zhang, Jinghui Wang, Wenhao Zhuang, Zheng Lin, Huiming Wang, Zhaoxiang Zhang, Yuqun Zhang, Haotian Zhang, Bin Chen, Jiaheng Liu

机构 * Kuaishou Technology（快手科技） ； Nanjing University（南京大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Isolating Culture Neurons in Multilingual Large Language Models

Danial Namazifard, Lukas Galke Poech

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted at IJCNLP-AACL 2025

EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models

Jiacheng Huang, Ning Yu, Xiaoyin Yi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by AAAI2026

Evaluation Awareness Scales Predictably in Open-Weights Large Language Models

Maheep Chaudhary, Ian Su, Nikhil Hooda, Nishith Shankar, Julia Tan, Kevin Zhu, Ryan Lagasse, Vasu Sharma, Ashwinee Panda

机构 * Independent（独立研究者） ； UC Irvine（加州大学尔湾分校） ； University of Waterloo（滑铁卢大学） ； UW Madison（威斯康星大学麦迪逊分校） ； University of Toronto（多伦多大学） ； Algoverse（Algoverse公司） ； META ； University of Maryland（马里兰大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

A Comparative Approach to Assessing Linguistic Creativity of Large Language Models and Humans

Anca Dinu, Andra-Maria Florescu, Alina Resceanu

机构 * University of Bucharest（布加勒斯大学） ； Interdisciplinary School of Doctoral Studies（跨学科博士研究学校） ； University of Craiova（克里奥瓦大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted for presentation at KES 2025. To appear in Procedia Computer Science (Elsevier)

Journal ref Procedia Computer Science 270 (2025) 1292-1301

Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering

Jihao Zhao, Chunlai Zhou, Daixuan Li, Shuaishuai Zu, Biao Qin

机构 * School of Information, Renmin University of China（信息学院，中国人民大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(title)；small language model(abstract)；分类 cs.CL

The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Models

Claudia Herambourg, Dawid Siuda, Julia Kopczyńska, Joao R. L. Santos, Wojciech Sas, Joanna Śmietańska-Nowak

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

A Large Language Model for Corporate Credit Scoring

Chitro Majumdar, Sergio Scandizzo, Ratanlal Mahanta, Avradip Mandal, Swarnendu Bhattacharjee

机构 * RsRL (R-square RiskLab)（RsRL（R-square RiskLab）） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

What a diff makes: automating code migration with large language models

Katherine A. Rosenfeld, Cliff C. Kerr, Jessica Lundin

机构 * Institute for Disease Modeling, Gates Foundation（疾病模型研究所，盖茨基金会） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 10 pages, 8 figures

Predicate Renaming via Large Language Models

Elisabetta Gentili, Tony Ribeiro, Fabrizio Riguzzi, Katsumi Inoue

机构 * Nantes Université, École Centrale Nantes, CNRS, LS2N（南特大学，中央南特学院，CNRS，LS2N） ； National Institute of Informatics（日本信息机构） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Generative Large Language Models (gLLMs) in Content Analysis: A Practical Guide for Communication Research

Daria Kravets-Meinke, Hannah Schmid-Petri, Sonja Niemann, Ute Schmid

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector

Haoyan Yang, Runxue Bao, Cao Xiao, Jun Ma, Parminder Bhatia, Shangqian Gao, Taha Kass-Hout

机构 * New York University（纽约大学） ； GE Healthcare（通用电气医疗） ； Florida State University（佛罗里达州立大学） 纠错

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；prompting(abstract)

Comments Accepted at NeurIPS 2025 (Camera-Ready Version)

LangLingual: A Personalised, Exercise-oriented English Language Learning Tool Leveraging Large Language Models

Sammriddh Gupta, Sonit Singh, Aditya Joshi, Mira Kim

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 14 pages

Does In-IDE Calibration of Large Language Models work at Scale?

Roham Koohestani, Agnia Sergeyuk, David Gros, Claudio Spiess, Sergey Titov, Prem Devanbu, Maliheh Izadi

机构 * Delft University of Technology（代尔夫特理工大学） ； JetBrains Research（JetBrains研究） ； University of California, Davis（加州大学戴维斯分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Under Review

DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models

Cathy Jiao, Yijun Pan, Emily Xiao, Daisy Sheng, Niket Jain, Hanzhang Zhao, Ishita Dasgupta, Jiaqi W. Ma, Chenyan Xiong

机构 * Carnegie Mellon University（卡内基梅隆大学） ； University of Michigan（密歇根大学） ； UIUC（伊利诺伊大学香槟分校） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL

Comments NeurIPS 2025 Datasets and Benchmarks Track

ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area

Junxian Li, Di Zhang, Xunzhi Wang, Zeying Hao, Jingdi Lei, Qian Tan, Cai Zhou, Wei Liu, Yaotian Yang, Xinrui Xiong, Weiyun Wang, Zhe Chen, Wenhai Wang, Wei Li, Shufei Zhang, Mao Su, Wanli Ouyang, Yuqiang Li, Dongzhan Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments 11 pages, updated version

REvolution: An Evolutionary Framework for RTL Generation driven by Large Language Models

Kyungjun Min, Kyumin Cho, Junhwan Jang, Seokhyeong Kang

机构 * Department of Electrical Engineering, Pohang University of Science and Technology（电子工程系，釜山科学技术大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted for publication at the 2026 Asia and South Pacific Design Automation Conference (ASP-DAC)

TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models

Florian Tambon, Amin Nikanjam, Cyrine Zid, Foutse Khomh, Giuliano Antoniol

机构 * SnT, University of Luxembourg（卢森堡大学SnT研究所） ； Huawei Distributed Scheduling and Data Engine Lab（华为分布式调度与数据引擎实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted ACM Transactions on Software Engineering and Methodology

Towards Reliable Evaluation of Large Language Models for Multilingual and Multimodal E-Commerce Applications

Shuyi Xie, Ziqin Liew, Hailing Zhang, Haibo Zhang, Ling Hu, Zhiqiang Zhou, Shuman Liu, Anxiang Zeng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

"You Are Rejected!": An Empirical Study of Large Language Models Taking Hiring Evaluations

Dingjie Fu, Dianxing Shi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Technical Report, 14 pages, 8 figures

PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models

Farhan Farsi, Shayan Bali, Fatemeh Valeh, Parsa Ghofrani, Alireza Pakniat, Kian Kashfipour, Amir H. Payberah

机构 * Amirkabir University of Technology（阿米尔卡比尔技术大学） ； King’s College London（伦敦国王学院） ； Politecnico di Milano（米兰理工学院） ； KTH Royal Institute of Technology（皇家理工学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy

Jinghang Shi, Xiaoyu Tang, Yang Huang, Yuyang Li, Xiao Kong, Yanxia Zhang, Caizhan Yue

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

An Automated Multi-modal Evaluation Framework for Mobile Intelligent Assistants Based on Large Language Models and Multi-Agent Collaboration

Meiping Wang, Jian Zhong, Rongduo Han, Liming Kang, Zhengkun Shi, Xiao Liang, Xing Lin, Nan Gao, Haining Zhang

机构 * College of Software, Nankai University（南开大学软件学院） ； vivo AI Lab（vivo人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

PARALLELPROMPT: Extracting Parallelism from Large Language Model Queries

Steven Kolawole, Keshav Santhanam, Virginia Smith, Pratiksha Thaker

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Stanford University（斯坦福大学） 纠错

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；prompting(abstract)

Comments 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Datasets and Benchmarks Track

Evaluating Large Language Models on Urdu Idiom Translation

Muhammad Farmal Khan, Mousumi Akter

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comprehending Spatio-temporal Data via Cinematic Storytelling using Large Language Models

Panos Kalnis. Shuo Shang, Christian S. Jensen

机构 * King Abdullah University of Science and Technology (KAUST)（卡塔尔国王 Abdullah 科学与技术大学） ； University of Electronic Science and Technology of China (UESTC)（中国电子科学与技术大学） ； Aalborg University（奥尔堡大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 5 pages

Journal ref SSTD '25: Proceedings of the 19th International Symposium on Spatial and Temporal Data, Pages 12,26, 2025

Safeguarding Efficacy in Large Language Models: Evaluating Resistance to Human-Written and Algorithmic Adversarial Prompts

Tiarnaigh Downey-Webb, Olamide Jogunola, Oluwaseun Ajao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 10 pages, 4 pages manuscript submitted to the Language Resources and Evaluation Conference (LREC 2026)