语言大模型 / LLM

NOTAM-Evolve: A Knowledge-Guided Self-Evolving Optimization Framework with LLMs for NOTAM Interpretation

Maoqi Liu, Quan Fang, Yuhao Wu, Can Zhao, Yang Yang, Kaiquan Cai

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to AAAI 2026

Revisiting NLI: Towards Cost-Effective and Human-Aligned Metrics for Evaluating LLMs in Question Answering

Sai Shridhar Balamurali, Lu Cheng

机构 * University of Illinois at Chicago（伊利诺伊大学香槟分校） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

LLMs vs. Traditional Sentiment Tools in Psychology: An Evaluation on Belgian-Dutch Narratives

Ratna Kandala, Katie Hoemann

专题命中 评测与基准 ：LLM(abstract,comments)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments NeurIPS 2025 Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion

Samet Hicsonmez, Abd El Rahman Shabayek, Djamila Aouada

机构 * University of Luxembourg（卢森堡大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)

Comments WACV 2026

DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation

Vishal Kumar, Shubhra Mishra, Rebecca Hao, Rizwaan Malik, David Broman, Dorottya Demszky

机构 * Stanford University（斯坦福大学） ； KTH Royal Institute of Technology（皇家理工学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Published at the Math-AI Workshop at NeurIPS 2025

Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring

Wenhao Xu, Akshatha Arodi, Jian-Yun Nie, Arsene Fansi Tchango

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments To appear at AAAI-26 (Social Impact Track)

AIA Forecaster: Technical Report

Rohan Alur, Bradly C. Stadie, Daniel Kang, Ryan Chen, Matt McManus, Michael Rickert, Tyler Lee, Michael Federici, Richard Zhu, Dennis Fogerty, Hayley Williamson, Nina Lozinski, Aaron Linsky, Jasjeet S. Sekhon

机构 * Bridgewater AIA Labs（布里奇沃特AIA实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Decoding Latent Attack Surfaces in LLMs: Prompt Injection via HTML in Web Summarization

Ishaan Verma, Arsheya Yadav

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Question Answering

Larissa Pusch, Tim O. F. Conrad

机构 * Zuse Institute Berlin（柏林泽尼茨研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents

Manasi Sharma, Chen Bo Calvin Zhang, Chaithanya Bandi, Clinton Wang, Ankit Aich, Huy Nghiem, Tahseen Rabbani, Ye Htet, Brian Jang, Sumana Basu, Aishwarya Balwani, Denis Peskoff, Marcos Ayestaran, Sean M. Hendryx, Brad Kenstler, Bing Liu

机构 * Scale AI ； University of Maryland（马里兰大学） ； University of Chicago（芝加哥大学） ； Washington University, St. Louis（圣路易斯华盛顿大学） ； McGill University（麦吉尔大学） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 21 figures, pre-print

Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QA

Sher Badshah, Hassan Sajjad

机构 * Faculty of Computer Science（计算机科学学院） ； Dalhousie University（达尔豪斯大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the 9th Widening NLP Workshop

Hierarchical structure understanding in complex tables with VLLMs: a benchmark and experiments

Luca Bindini, Simone Giovannini, Simone Marinai, Valeria Nardoni, Kimiya Noor Ali

机构 * DINFO -- University of Florence, Florence, Italy（DINFO -- 佛罗伦萨大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis

Rochana R. Obadage, Sarah M. Rajtmajer, Jian Wu

机构 * Old Dominion University（旧 Dominion 大学） ； The Pennsylvania State University（宾夕法尼亚州立大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Peer reviewed and accepted at JCDL 2025, 16 pages, 7 figures

Auto-US: An Ultrasound Video Diagnosis Agent Using Video Classification Framework and LLMs

Yuezhe Yang, Yiyue Guo, Wenjie Cai, Qingqing Ruan, Siying Wang, Xingbo Dong, Zhe Jin, Yong Dai

机构 * The Anhui Provincial International Joint Research Center for Advanced Technology in Medical Imaging（安徽省级国际联合先进技术医学影像研究中心） ； Anhui University（安徽大学） ； School of First Clinical Medicine（第一临床医学院） ； Anhui University of Science and Technology（安徽理工大学） ； School of Medicine（医学院） ； The First Hospital（第一医院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Under Review

SugarTextNet: A Transformer-Based Framework for Detecting Sugar Dating-Related Content on Social Media with Context-Aware Focal Loss

Lionel Z. Wang, Shihan Ben, Yulu Huang, Simeng Qin

机构 * The Hong Kong Polytechnic University（香港理工大学） ； The University of Hong Kong（香港大学） ； Northeastern University（东北大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments This paper is accepted by HICSS 2026

WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada

Braeden Sherritt, Isar Nejadgholi, Efstratios Aivaliotis, Khaled Mslmani, Marzieh Amini

机构 * Carleton University（卡尔顿大学） ； National Research Council Canada（加拿大国家研究委员会） 纠错

专题命中 评测与基准 ：language model(abstract)；prompting(abstract)；分类 cs.AI

Integrating Artificial Intelligence into Operating Systems: A Survey on Techniques, Applications, and Future Directions

Yifan Zhang, Xinkui Zhao, Ziying Li, Guanjie Cheng, Jianwei Yin, Lufei Zhang, Zuoning Chen

机构 * School of Software Technology, Zhejiang University（浙江大学软件学院） ； College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； State Key Laboratory of Mathematical Engineering and Advanced Computing（数学工程与先进计算国家重点实验室） ； Chinese Academy of Engineering（中国工程院） ； College of Cyber Security, Jinan University（暨南大学网络安全学院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 68 pages,9 figures

Digital Nature Revisited: A Ten-Year Synthesis of Art, Technology, and the Evolution of "Nature": Reimagining Post-Truth Ecologies Through Art, Algorithm, and Animism

Yoichi Ochiai, Takashi Shimizu

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments 9 pages

Revisiting MLLM Based Image Quality Assessment: Errors and Remedy

Zhenchen Tang, Songlin Yang, Bo Peng, Zichuan Wang, Jing Dong

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments 13 pages

DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry

Changti Wu, Shijie Lian, Zihao Liu, Lei Zhang, Laurence Tianruo Yang, Kai Chen

机构 * East China Normal University（华东师范大学） ； Zhongguancun Academy（中关村学院） ； Huazhong University of Science and Technology（华中科技大学） ； Peking University（北京大学） ； Zhengzhou University（郑州大学） ； Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The code and dataset are available at \href{https://zgca-ai4edu.github.io/DynaSolidGeo/}{DynaSolidGeo}

Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages

Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, Stephen H. Bach

机构 * Brown University（布朗大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL Main Conference 2025

Journal ref Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

COPA: Comparing the incomparable in multi-objective model evaluation

Adrián Javaloy, Antonio Vergari, Isabel Valera

机构 * University of Edinburgh（爱丁堡大学） ； Saarland University（萨尔兰大学） 纠错

专题命中 评测与基准 ：foundation model(abstract)；分类 cs.AI、cs.LG

Comments 29 pages, 18 figures. Under submission

How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity

Zihan Ma, Dongsheng Zhu, Shudong Liu, Taolin Zhang, Junnan Liu, Qingqiu Li, Minnan Luo, Songyang Zhang, Kai Chen

机构 * School of Computer Science and Technology Xi’an Jiaotong University China（西安交通大学计算机科学与技术学院） ； Shanghai AI Laboratory（上海人工智能实验室） ； MOE KLINNS Lab Xi’an Jiaotong University China（西安交通大学教育部KLINNS实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.CL

Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis

Aditi Singhania, Krutik Malani, Riddhi Dhawan, Arushi Jain, Garv Tandon, Nippun Sharma, Souymodip Chakraborty, Vineet Batra, Ankit Phogat

专题命中 评测与基准 ：prompting(abstract)；分类 cs.AI

Towards a Standard, Enterprise-Relevant Agentic AI Benchmark: Lessons from 5.5 billion tokens' worth of agentic AI evaluations

JV Roig

机构 * Kamiwaza AI 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.AI

Comments 34 pages

CellARC: Measuring Intelligence with Cellular Automata

Miroslav Lžičař

专题命中 评测与基准 ：LLM(abstract)；分类 cs.LG

Comments 22 pages, 11 figures. Working draft. Dataset and leaderboard available at https://cellarc.mireklzicar.com

Active Learning for Animal Re-Identification with Ambiguity-Aware Sampling

Depanshu Sani, Mehar Khurana, Saket Anand

专题命中 评测与基准 ：foundation model(abstract)；分类 cs.AI

Comments In Proceedings of AAAI Conference on Artificial Intelligence 2026

How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective

Songsong Yu, Yuxin Chen, Hao Ju, Lianjie Jia, Fuxi Zhang, Shaofei Huang, Yuhan Wu, Rundi Cui, Binghao Ran, Zaibin Zhang, Zhedong Zheng, Zhipeng Zhang, Yifan Wang, Lin Song, Lijun Wang, Yanwei Li, Ying Shan, Huchuan Lu

机构 * ARC Lab, Tencent PCG（腾讯PCG部门ARC实验室） ； Shanghai Jiao Tong University（上海交通大学） ； University of Macau（澳门大学） ； Dalian University of Technology（大连理工大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.AI

Comments a comprehensive visual spatial reasoning evaluation tool, 25 pages, 16 figures

SOCIA-$\nabla$: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generation

Yuncheng Hua, Sion Weatherhead, Mehdi Jafari, Hao Xue, Flora D. Salim

机构 * School of Computer Science Engineering, University of New South Wales（计算机科学工程学院，新南威尔士大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.AI

Comments 11 pages, 1 figure, 2 tables. The paper is under review

3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark

Ivan Sviridov, Amina Miftakhova, Artemiy Tereshchenko, Galina Zubkova, Pavel Blinov, Andrey Savchenko

机构 * Sber AI Lab（Sber AI实验室） ； HSE University（俄罗斯高等经济大学） ； ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息与系统问题研究所可信人工智能研究中心） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL

Comments EMNLP 25 (main)

Journal ref https://aclanthology.org/2025.emnlp-main.1353/