语言大模型 / LLM

Large Language Model Agent Personality and Response Appropriateness: Evaluation by Human Linguistic Experts, LLM-as-Judge, and Natural Language Processing Model

Eswari Jayakumar, Niladri Sekhar Dash, Debasmita Mukherjee

专题命中 评测与基准 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

ChessQA: Evaluating Large Language Models for Chess Understanding

Qianfeng Wen, Zhenwei Tang, Ashton Anderson

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments 33 pages,8 figures

Arena-Lite: Efficient and Reliable Large Language Model Evaluation via Tournament-Based Direct Comparisons

Seonil Son, Ju-Min Oh, Heegon Jin, Cheolhun Jang, Jeongbeom Jeong, Kuntae Kim

机构 * NC AI ； RLWRLD Inc.（RLWRLD公司） ； Samsung AI Research（三星人工智能研究所） ； Global AI Platform（全球人工智能平台） ； Samsung Life Insurance（三星人寿保险） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 8 pages for main body, 19 pages in total

Journal ref EMNLP 2025 Main

Discovering Heuristics with Large Language Models (LLMs) for Mixed-Integer Programs: Single-Machine Scheduling

İbrahim Oğuz Çetinkaya, İ. Esra Büyüktahtakın, Parshin Shojaee, Chandan K. Reddy

机构 * Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA（弗吉尼亚理工大学工业与系统工程系） ； Department of Computer Science, Virginia Tech, Arlington, VA（弗吉尼亚理工大学计算机科学系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs

Soham Satyadharma, Fatemeh Sheikholeslami, Swati Kaul, Aziz Umit Batur, Suleiman A. Khan

机构 * Amazon Catalog AI（亚马逊目录人工智能） 纠错

专题命中 评测与基准 ：prompting(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Pre-trained knowledge elevates large language models beyond traditional chemical reaction optimizers

Robert MacKnight, Jose Emilio Regio, Jeffrey G. Ethier, Luke A. Baldwin, Gabe Gomes

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 27 pages, 8 figures

Generative Large Language Models (gLLMs) in Content Analysis: A Practical Guide for Communication Research

Daria Kravets-Meinke, Hannah Schmid-Petri, Sonja Niemann, Ute Schmid

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector

Haoyan Yang, Runxue Bao, Cao Xiao, Jun Ma, Parminder Bhatia, Shangqian Gao, Taha Kass-Hout

机构 * New York University（纽约大学） ； GE Healthcare（通用电气医疗） ； Florida State University（佛罗里达州立大学） 纠错

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；prompting(abstract)

Comments Accepted at NeurIPS 2025 (Camera-Ready Version)

Evaluating the effectiveness of LLM-based interoperability

Rodrigo Falcão, Stefan Schweitzer, Julien Siebert, Emily Calvet, Frank Elberzhager

机构 * Fraunhofer IESE（弗劳恩霍夫研究所IESE） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

SANSKRITI: A Comprehensive Benchmark for Evaluating Language Models' Knowledge of Indian Culture

Arijit Maji, Raghvendra Kumar, Akash Ghosh, Anushka, Sriparna Saha

机构 * Department of Computer Science and Engineering, Indian Institute of Technology Patna（印度理工学院帕纳布分校计算机科学与工程系） ； Department of Computer Science, Banasthali Vidyapeeth University（班斯塔利大学计算机科学系） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；small language model(abstract)；分类 cs.CL

Comments ACL 2025 Findings

LLM-as-a-Judge for Software Engineering: Literature Review, Vision, and the Road Ahead

Junda He, Jieke Shi, Terry Yue Zhuo, Christoph Treude, Jiamou Sun, Zhenchang Xing, Xiaoning Du, David Lo

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Can LLMs Write Faithfully? An Agent-Based Evaluation of LLM-generated Islamic Content

Abdullah Mushtaq, Rafay Naeem, Ezieddin Elmahjub, Ibrahim Ghaznavi, Shawqi Al-Maliki, Mohamed Abdallah, Ala Al-Fuqaha, Junaid Qadir

机构 * Information Technology University（信息科技大学） ； Qatar University（卡塔尔大学） ； Hamad Bin Khalifa University（哈马德·本·卡西姆大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: 5th Muslims in Machine Learning (MusIML) Workshop

Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual Augmentation

Kaveh Eskandari Miandoab, Mahammed Kamruzzaman, Arshia Gharooni, Gene Louis Kim, Vasanth Sarathy, Ninareh Mehrabi

机构 * Tufts University（塔夫茨大学） ； University of South Florida（佛罗里达州立大学） ； Sharif University of Technology（谢赫·穆吉布技术大学） ； Meta ； Resolution（解决方案） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 3 figures, 3 tables

MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Revision

Yuyang Wu, Jinhui Ye, Shuhao Zhang, Lu Dai, Yonatan Bisk, Olexandr Isayev

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Hong Kong University of Science and Technology（香港科学与技术大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 9 pages

Journal ref EMNLP2025

From Time and Place to Preference: LLM-Driven Geo-Temporal Context in Recommendations

Yejin Kim, Shaghayegh Agah, Mayur Nankani, Neeraj Sharma, Feifei Peng, Maria Peifer, Sardar Hamidian, H Howie Huang

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

Banal Deception Human-AI Ecosystems: A Study of People's Perceptions of LLM-generated Deceptive Behaviour

Xiao Zhan, Yifan Xu, Noura Abdi, Joe Collenette, Ruba Abu-Salma, Stefan Sarkadi

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

Journal ref Journal of Artificial Intelligence Research84, Article 11(October 2025)

Reinforcement Learning for Long-Horizon Multi-Turn Search Agents

Vivek Kalyan, Martin Andrews

机构 * Red Cat Labs, Singapore（红猫实验室（新加坡）） ； Singapore（新加坡） 纠错

专题命中 评测与基准 ：large language model(abstract,comments)；language model(abstract,comments)；LLM(abstract)；分类 cs.CL

Comments 4 pages plus references and appendices. Accepted into the First Workshop on Multi-Turn Interactions in Large Language Models at NeurIPS 2025

Evaluation of Geographical Distortions in Language Models

Rémy Decoupes, Roberto Interdonato, Mathieu Roche, Maguelonne Teisseire, Sarah Valentin

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Comments Accepted version. Published in Machine Learning (Springer) 114:263 (2025). Open access under a CC BY-NC-ND 4.0 license. DOI: 10.1007/s10994-025-06916-9

Journal ref Machine Learning (2025) 114:263

A word association network methodology for evaluating implicit biases in LLMs compared to humans

Katherine Abramski, Giulio Rossetti, Massimo Stella

机构 * University of Pisa, Department of Computer Science（比萨大学计算机科学系） ； National Research Council of Italy, Institute of Information Science and Technologies（意大利国家研究委员会信息科学与技术研究所） ； University of Trento, Department of Psychology and Cognitive Science（特伦托大学心理学与认知科学系） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 13 figures, 3 tables

Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Korean

Chanwoo Park, Suyoung Park, JiA Kang, Jongyeon Park, Sangho Kim, Hyunji M. Park, Sumin Bae, Mingyu Kang, Jaejin Lee

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments submitted to ACL ARR Rolling Review

Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs

Jyotika Singh, Weiyi Sun, Amit Agarwal, Viji Krishnamurthy, Yassine Benajiba, Sujith Ravi, Dan Roth

机构 * Oracle AI 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2025

MixAT: Combining Continuous and Discrete Adversarial Training for LLMs

Csaba Dékány, Stefan Balauca, Robin Staab, Dimitar I. Dimitrov, Martin Vechev

机构 * INSAIT, Sofia University "St. Kliment Ohridski"（INSAIT，索菲亚大学"圣克莱孟·奥赫里迪斯基"） ； ETH Zurich（苏黎世联邦理工学院） ； ELTE Eötvös Loránd University, Budapest, Hungary（ELTE 爱斯特霍特·洛兰德大学，布达佩斯，匈牙利） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published at 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic

Yuxuan Sun, Yixuan Si, Chenglu Zhu, Kai Zhang, Zhongyi Shui, Bowen Ding, Tao Lin, Lin Yang

机构 * College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院） ； Research Center for Industries of the Future and School of Engineering, Westlake University, China（未来产业研究中心和西湖大学工程学院） ； Department of Computer Science and Engineering, The Ohio State University, USA（俄亥俄州立大学计算机科学与工程系） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)

Comments 52 pages, 34 figures

From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning

Zihan Chen, Song Wang, Xingbo Fu, Chengshuai Shi, Zhenyu Lei, Cong Shen, Jundong Li

机构 * Department of ECE, University of Virginia（电子工程系，弗吉尼亚大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions

Cheng Luo, Jianghui Wang, Bing Li, Siyang Song, Bernard Ghanem

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 25 pages, 9 figures

Evaluating LLMs on Generating Age-Appropriate Child-Like Conversations

Syed Zohaib Hassan, Pål Halvorsen, Miriam S. Johnson, Pierre Lison

机构 * Department of Holistic Systems, SimulaMet（整体系统系，SimulaMet） ； Department of Computer Science, Oslo Metropolitan University（计算机科学系，奥斯陆 Metropolitan 大学） ； Department of Behavioural Sciences, Oslo Metropolitan University（行为科学系，奥斯陆 Metropolitan 大学） ； Department of Psychology, Harvard University（心理学系，哈佛大学） ； Department of Informatics, University of Oslo（信息学系，奥斯陆大学） ； Norwegian Computing Center（挪威计算中心） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 11 pages excluding references and appendix. 3 figures and 6 tables

Challenging Multilingual LLMs: A New Taxonomy and Benchmark for Unraveling Hallucination in Translation

Xinwei Wu, Heng Liu, Jiang Zhou, Xiaohu Zhao, Linlong Xu, Longyue Wang, Weihua Luo, Kaifu Zhang

机构 * Tianjin University（天津大学） ； Alibaba International Digital Commerce（阿里巴巴国际数字商业） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

PULSE: Practical Evaluation Scenarios for Large Multimodal Model Unlearning

Tatsuki Kawakami, Kazuki Egashira, Atsuyuki Miyai, Go Irie, Kiyoharu Aizawa

机构 * The University of Tokyo（东京大学） ； Tokyo University of Science（东京科学大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG；LLM(comments)

Comments Accepted at NeurIPS 2025 Workshop: Evaluating the Evolving LLM Lifecycle

LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability

Zikai Xiao, Fei Huang, Jianhong Tu, Jianhui Wei, Wen Ma, Yuxuan Zhou, Jian Wu, Bowen Yu, Zuozhu Liu, Junyang Lin

机构 * Zhejiang University（浙江大学） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP Findings 2025

A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications

Minhua Lin, Zongyu Wu, Zhichao Xu, Hui Liu, Xianfeng Tang, Qi He, Charu Aggarwal, Hui Liu, Xiang Zhang, Suhang Wang

机构 * The Pennsylvania State University（宾夕法尼亚州立大学） ； The University of Utah（犹他大学） ； Amazon（亚马逊） ； Microsoft（微软） ； IBM T.J. Watson Research Center（IBM T.J.沃森研究中心） ； Michigan State University（密歇根州立大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 38 pages, 4 figures, 7 tables