语言大模型 / LLM

Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications

Guangxin Su, Hanchen Wang, Jianwei Wang, Wenjie Zhang, Ying Zhang, Jian Pei

机构 * The University of New South Wales（新南威尔士大学） ； The University of Technology Sydney（技术大学悉尼分校） ； University of Technology Sydney（技术大学悉尼分校） ； Duke University（杜克大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Surveys and overviews; Natural language processing; Knowledge representation and reasoning; Graph algorithms

Does Model Size Matter? A Comparison of Small and Large Language Models for Requirements Classification

Mohammad Amin Zadenoori, Vincenzo De Martino, Jacek Dabrowski, Xavier Franch, Alessio Ferrari

机构 * University of Padova（帕多瓦大学） ； Software Engineering (SeSa) Lab, University of Salerno（软件工程（SeSa）实验室，萨勒诺大学） ； Universitat Politècnica de Catalunya（加泰罗尼亚理工大学） ； Lero, the Research Ireland Centre for Software, University of Limerick（爱尔兰软件研究中心（Lero）, 奥斯曼大学） ； University College Dublin (UCD)（都柏林大学学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；small language model(abstract)；分类 cs.CL、cs.AI

LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization

Zhen-Song Chen, Hong-Wei Ding, Xian-Jia Wang, Witold Pedrycz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments This work has been submitted to the IEEE for possible publication

Do Large Language Models Know How Much They Know?

Gabriele Prato, Jerry Huang, Prasanna Parthasarathi, Shagun Sodhani, Sarath Chandar

机构 * Chandar Research Lab（昌达尔研究实验室） ； Mila – Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； Meta FAIR ； Polytechnique Montréal（蒙特利尔理工学院） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a long paper at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Official version of paper within conference proceedings is available at https://aclanthology.org/2024.emnlp-main.348/

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, Ran He

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室） ； School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院） ； Tencent Youtu Lab（腾讯优图实验室） ； Xiamen University（厦门大学） ； CASIA（中国科学院自动化研究所） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments NeurIPS DB 2025 Spotlight, Project Page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation

REvolution: An Evolutionary Framework for RTL Generation driven by Large Language Models

Kyungjun Min, Kyumin Cho, Junhwan Jang, Seokhyeong Kang

机构 * Department of Electrical Engineering, Pohang University of Science and Technology（电子工程系，釜山科学技术大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted for publication at the 2026 Asia and South Pacific Design Automation Conference (ASP-DAC)

TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models

Florian Tambon, Amin Nikanjam, Cyrine Zid, Foutse Khomh, Giuliano Antoniol

机构 * SnT, University of Luxembourg（卢森堡大学SnT研究所） ； Huawei Distributed Scheduling and Data Engine Lab（华为分布式调度与数据引擎实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments Accepted ACM Transactions on Software Engineering and Methodology

Can large audio language models understand child stuttering speech? speech summarization, and source separation

Chibuzor Okocha, Maya Bakri, Christan Grant

机构 * Department of Computer Science, University of Florida, Gainesville, FL, USA（佛罗里达大学计算机科学系） ； Department of Computer Science, Lebanese American University（黎巴嫩美国大学计算机科学系） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL

Comments 7 pages, 1 Figure, 8 tables, Under review ICASSP 2026

Save It for the "Hot" Day: An LLM-Empowered Visual Analytics System for Heat Risk Management

Haobo Li, Wong Kam-Kwai, Yan Luo, Juntong Chen, Chengzhong Liu, Yaxuan Zhang, Alexis Kai Hon Lau, Huamin Qu, Dongyu Liu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

SAMA: Towards Multi-Turn Referential Grounded Video Chat with Large Language Models

Ye Sun, Hao Zhang, Henghui Ding, Tiehua Zhang, Xingjun Ma, Yu-Gang Jiang

专题命中 评测与基准 ：large language model(title)；language model(title)

Comments NeurIPS 2025

MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges?

Yunxiang Zhang, Muhammad Khalifa, Shitanshu Bhushan, Grant D Murphy, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, Lu Wang

机构 * University of Michigan（密歇根大学） ； LG AI Research（LG人工智能研究） ； University of Illinois at Chicago（伊利诺伊大学香槟分校） 纠错

专题命中 评测与基准 ：language agent(title,abstract)；LLM(abstract)；分类 cs.AI

Comments NeurIPS 2025 Datasets and Benchmarks Track

Irish-BLiMP: A Linguistic Benchmark for Evaluating Human and Language Model Performance in a Low-Resource Setting

Josh McGiff, Khanh-Tung Tran, William Mulcahy, Dáibhidh Ó Luinín, Jake Dalzell, Róisín Ní Bhroin, Adam Burke, Barry O'Sullivan, Hoang D. Nguyen, Nikola S. Nikolov

机构 * University of Limerick（利默里克大学） ； University College Cork（科克大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 8 pages

Federated Low-Rank Adaptation for Foundation Models: A Survey

Yiyuan Yang, Guodong Long, Qinghua Lu, Liming Zhu, Jing Jiang, Chengqi Zhang

机构 * University of Technology Sydney（技术大学悉尼大学） ； CSIRO’s Data61（CSIRO的数据61） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.LG

Journal ref Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-25), Survey Track, pp. 10779-10787, 2025

Soppia: A Structured Prompting Framework for the Proportional Assessment of Non-Pecuniary Damages in Personal Injury Cases

Jorge Alberto Araujo

机构 * Tribunal Regional do Trabalho da 4ª Região (TRT4)（第四地区劳动法庭） 纠错

专题命中 评测与基准 ：prompting(title,abstract)；分类 cs.AI

Comments 9 pages, 2 tables, includes GitHub link to framework implementation. Submitted to the Artificial Intelligence and Law section of arXiv

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents

Qinghua Lu, Dehai Zhao, Yue Liu, Hao Zhang, Liming Zhu, Xiwei Xu, Angela Shi, Tristan Tan, Rick Kazman

机构 * Data61, CSIRO, Australia（Data61，CSIRO，澳大利亚） ； Empathetic AI, Australia（Empathetic AI，澳大利亚） ； University of Hawaii, Honolulu, HI, USA（夏威夷大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.AI

TripTide: A Benchmark for Adaptive Travel Planning under Disruptions

Priyanshu Karmakar, Soumyabrata Chaudhuri, Shubhojit Mallick, Manish Gupta, Abhik Jana, Shreya Ghosh

机构 * School of Electrical and Computer Sciences, IIT Bhubaneswar（电子与计算机科学学院，印度理工学院Bhubaneswar分校） ； Microsoft（微软） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 12 tables and 7 figures

RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback

Chunyu Miao, Henry Peng Zou, Yangning Li, Yankai Chen, Yibo Wang, Fangxin Wang, Yifan Li, Wooseong Yang, Bowei He, Xinni Zhang, Dianzhi Yu, Hanchen Yang, Hoang H Nguyen, Yue Zhou, Jie Yang, Jizhou Guo, Wenzhe Fan, Chin-Yuan Yeh, Panpan Meng, Liancheng Fang, Jinhu Qi, Wei-Chieh Huang, Zhengyao Gu, Yuwei Han, Langzhou He, Yuyao Yang, Yinghui Li, Hai-Tao Zheng, Xue Liu, Irwin King, Philip S. Yu

机构 * University of Illinois Chicago（伊利诺伊大学芝加哥分校） ； Tsinghua University（清华大学） ； McGill University（麦吉尔大学） ； MBZUAI（麦吉尔大学人工智能研究所） ； The Chinese University of Hong Kong（香港中文大学） ； City University of Hong Kong（香港城市大学） ； Shanghai Jiao Tong University（上海交通大学） ； National Taiwan University（国立台湾大学） ； Xi’an Jiaotong University（西安交通大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Code and dataset are available at github.com/ChunyuMiao98/RECODE

Influence Guided Context Selection for Effective Retrieval-Augmented Generation

Jiale Deng, Yanyan Shen, Ziyuan Pei, Youmin Chen, Linpeng Huang

机构 * Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

On Optimal Steering to Achieve Exact Fairness

Mohit Sharma, Amit Jayant Deshpande, Chiranjib Bhattacharyya, Rajiv Ratn Shah

机构 * Department of Computer Science IIIT Delhi, India（印度德里IIIT计算机科学系） ； Microsoft Research India（微软印度研究院） ； Department of Computer Science and Automation Indian Institute of Science, Bengaluru, India（印度班加罗尔印度科学学院计算机科学与自动化系） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted for Presentation at Neurips 2025

Risk Management for Mitigating Benchmark Failure Modes: BenchRisk

Sean McGregor, Victor Lu, Vassil Tashev, Armstrong Foundjem, Aishwarya Ramasethu, Sadegh AlMahdi Kazemi Zarkouei, Chris Knotz, Kongtao Chen, Alicia Parrish, Anka Reuel, Heather Frase

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 19 pages, 7 figures, to be published in the 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

Redefining Retrieval Evaluation in the Era of LLMs

Giovanni Trappolini, Florin Cuconasu, Simone Filice, Yoelle Maarek, Fabrizio Silvestri

机构 * Sapienza University of Rome（萨皮恩扎罗马大学） ； Technology Innovation Institute（技术创新研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Gen-Review: A Large-scale Dataset of AI-Generated (and Human-written) Peer Reviews

Luca Demetrio, Giovanni Apruzzese, Kathrin Grosse, Pavel Laskov, Emil Lupu, Vera Rimmer, Philine Widmer

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

L^2M^3OF: A Large Language Multimodal Model for Metal-Organic Frameworks

Jiyu Cui, Fang Wu, Haokai Zhao, Minggao Feng, Xenophon Evangelopoulos, Andrew I. Cooper, Yejin Choi

机构 * Department of Chemistry, University of Liverpool（利兹大学化学系） ； Leverhulme Research Centre for Functional Materials Design, University of Liverpool（利兹大学功能性材料设计研究所以） ； Department of Computer Science, University of Stanford（斯坦福大学计算机科学系） ； School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 18 pages, 7 figures

HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks

Stefan Abi-Karam, Cong Hao

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Cultural Alien Sampler: Open-ended art generation balancing originality and coherence

Alejandro H. Artiles, Hiromu Yakura, Levin Brinkmann, Mar Canet Sola, Hassan Abu Alhaija, Ignacio Serna, Nasim Rahaman, Bernhard Schölkopf, Iyad Rahwan

机构 * Max Planck Institute for Human Development（马克斯·普朗克人类发展研究所） ； Academy of Media Arts Cologne（科隆媒体艺术学院） ； Tiptree Systems（蒂普特里系统） ； NVIDIA（NVIDIA公司） ； BFM, Tallinn University（BFM，塔林大学） ； Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS 2025). Creative AI Track. 26 pages, 24 figures

ColorAgent: Building A Robust, Personalized, and Interactive OS Agent

Ning Li, Qiqiang Lin, Zheng Wu, Xiaoyun Mo, Weiming Zhang, Yin Zhao, Xiangmou Qu, Jiamu Zhou, Jun Wang, Congmin Zheng, Yuanyi Song, Hongjiang Chen, Heyuan Huang, Jihong Wang, Jiaxin Yin, Jingwei Yu, Junwei Liao, Qiuying Peng, Xingyu Lou, Jun Wang, Weiwen Liu, Zhuosheng Zhang, Weinan Zhang

机构 * Shanghai Jiao Tong University（上海交通大学） ； OPPO Research Institute（OPPO研究院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Misspellings in Natural Language Processing: A survey

Gianluca Sperduti, Alejandro Moreo

机构 * Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche（信息科学与技术研究所，国家研究理事会） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Customizing Open Source LLMs for Quantitative Medication Attribute Extraction across Heterogeneous EHR Systems

Zhe Fei, Mehmet Yigit Turali, Shreyas Rajesh, Xinyang Dai, Huyen Pham, Pavan Holur, Yuhui Zhu, Larissa Mooney, Yih-Ing Hser, Vwani Roychowdhury

机构 * Department of Statistics, UC Riverside（统计学系，加州大学河滨分校） ； Department of Electrical and Computer Engineering, UCLA（电气与计算机工程系，加州大学洛杉矶分校） ； Department of Psychiatry and Biobehavioral Sciences, UCLA（精神病学与生物行为科学系，加州大学洛杉矶分校） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2025: The Second Workshop on GenAI for Health: Potential, Trust, and Policy Compliance

T1: A Tool-Oriented Conversational Dataset for Multi-Turn Agentic Planning

Amartya Chakraborty, Paresh Dashore, Nadia Bathaee, Anmol Jain, Anirban Das, Shi-Xiong Zhang, Sambit Sahu, Milind Naphade, Genta Indra Winata

机构 * Capital One 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2025 Datasets and Benchmarks Track

Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation

Zhenghao Zeng, David Arbour, Avi Feller, Ishita Dasgupta, Atanu R Sinha, Edward H. Kennedy

专题命中 评测与基准 ：language model(title)

Comments 27 pages, 3 figures