语言大模型 / LLM

Classic4Children: Adapting Chinese Literary Classics for Children with Large Language Model

Jiali Chen, Xusen Hei, Yuqi Xue, Zihan Wu, Jiayuan Xie, Yi Cai

机构 * Key Laboratory of Big Data and Intelligent Robot (South China University of Technology) Ministry of Education（大数据与智能机器人重点实验室（华南理工大学）教育部） ； School of Software Engineering, South China University of Technology（软件学院（华南理工大学）） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comments Accepted at NAACL 2025 Findings

Help or Hurdle? Rethinking Model Context Protocol-Augmented Large Language Models

Wei Song, Haonan Zhong, Ziqi Ding, Jingling Xue, Yuekang Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play

Lucia Cipolina-Kun, Marianna Nezhurina, Jenia Jitsev

机构 * LAION Juelich Supercomputing Center (JSC)（LAION尤利奇超级计算机中心（JSC）） ； Research Center Juelich (FZJ)（尤利奇研究中心（FZJ）） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Cryfish: On deep audio analysis with Large Language Models

Anton Mitrofanov, Sergei Novoselov, Tatiana Prisyach, Vladislav Marchevskiy, Arseniy Karelin, Nikita Khmelev, Dmitry Dutov, Stepan Malykh, Igor Agafonov, Aleksandr Nikitin, Oleg Petrov

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Journal ref Proc. Interspeech 2025

GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Hans-Arno Jacobsen

机构 * University of Toronto（多伦多大学） 纠错

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；分类 cs.AI

Comments 12 pages, 5 figures

Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection

Zhipeng Wei, Yuqi Liu, N. Benjamin Erichson

机构 * International Computer Science Institute, CA, USA（国际计算机科学研究所） ； Lawrence Berkeley National Laboratory, CA, USA（伯克利国家实验室） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution

Zailong Tian, Zhuoheng Han, Yanzhe Chen, Haozhe Xu, Xi Yang, Richeng Xuan, Houfeng Wang, Lizi Liao

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Listening with Language Models: Using LLMs to Collect and Interpret Classroom Feedback

Sai Siddartha Maram, Ulia Zaman, Magy Seif El-Nasr

机构 * University of California, Santa Cruz（加州大学圣克鲁兹分校） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.AI

CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integration

Zixin Chen, Jiachen Wang, Yumeng Li, Haobo Li, Chuhan Shi, Rong Zhang, Huamin Qu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Journal ref ACM UIST 2025

Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset

Zhipeng Xue, Xiaoting Zhang, Zhipeng Gao, Xing Hu, Shan Gao, Xin Xia, Shanping Li

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Deep Language Geometry: Constructing a Metric Space from LLM Weights

Maksym Shamrai, Vladyslav Hamolia

机构 * Institute of Mathematics of NASU（国家科学院数学研究所） ； MacPaw 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 18 pages, accepted to RANLP 2025

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

David Heineman, Valentin Hofmann, Ian Magnusson, Yuling Gu, Noah A. Smith, Hannaneh Hajishirzi, Kyle Lo, Jesse Dodge

机构 * Allen Institute for Artificial Intelligence（艾伦人工智能研究所） ； Paul G. Allen School of Computer Science & Engineering, University of Washington（保罗·G·艾伦计算机科学与工程学院，华盛顿大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Deep Learning Model for Amyloidogenicity Prediction using a Pre-trained Protein LLM

Zohra Yagoub, Hafida Bouziane

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Towards Multimodal Social Conversations with Robots: Using Vision-Language Models

Ruben Janssens, Tony Belpaeme

机构 * Ghent University–imec（根特大学–imec） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL；foundation model(comments)

Comments Accepted at the workshop "Human - Foundation Models Interaction: A Focus On Multimodal Information" (FoMo-HRI) at IEEE RO-MAN 2025 (Camera-ready version)

MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation

Jianbo Dai, Jianqiao Lu, Yunlong Feng, Guangtao Zeng, Rongju Ruan, Ming Cheng, Dong Huang, Haochen Tan, Zhijiang Guo

机构 * University of Edinburgh（爱丁堡大学） ； The University of Hong Kong（香港大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Singapore University of Technology and Design（新加坡科技设计大学） ； South China University of Technology（华南理工大学） ； City University of Hong Kong（香港城市大学） ； University of Cambridge（剑桥大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 43 pages, dataset and code are available at https://github.com/SparksofAGI/MHPP, leaderboard can be found at https://sparksofagi.github.io/MHPP/

RUM: Rule+LLM-Based Comprehensive Assessment on Testing Skills

Yue Wang, Zhenyu Chen, Yuan Zhao, Chunrong Fang, Ziyuan Wang, Song Huang

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

TeleAntiFraud-28k: An Audio-Text Slow-Thinking Dataset for Telecom Fraud Detection

Zhiming Ma, Peidong Wang, Minhua Huang, Jingpeng Wang, Kai Wu, Xiangzhao Lv, Yachun Pang, Yin Yang, Wenjie Tang, Yuchen Kang

机构 * China Mobile Internet Company Ltd.（中国移动互联网有限公司） ； Northeastern University（东北大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Benchmark Dataset Generation and Evaluation for Excel Formula Repair with LLMs

Ananya Singha, Harshita Sahijwani, Walt Williams, Emmanuel Aboah Boateng, Nick Hausman, Miguel Di Luca, Keegan Choudhury, Chaya Binet, Vu Le, Tianwei Chen, Oryan Rokeah Chen, Sulaiman Vesal, Sadid Hasan

机构 * Microsoft（微软） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted at the KDD workshop on Evaluation and Trustworthiness of Agentic and Generative AI Models

What do Speech Foundation Models Learn? Analysis and Applications

Ankita Pasad

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.CL

Comments Ph.D. Thesis

LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data

Stephen Meisenbacher, Alexandra Klymenko, Florian Matthes

机构 * Technical University of Munich School of Computation, Information（慕尼黑技术大学计算、信息与技术学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Comments 13 pages, 3 figures, 4 tables. Accepted to HAIPS @ CCS 2025

Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challenges

Lajos Muzsai, David Imolai, András Lukács

机构 * Eötvös Loránd University, Institute of Mathematics, AI Research Group（埃奥武斯·洛尔兰大学数学学院人工智能研究组） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI

Comments 13 pages, 2 figures

Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality

Rahul Zalkikar, Kanchan Chandra

机构 * New York University（纽约大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pages 1337--1361

The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution

Elon Ezra, Ariel Weizman, Amos Azaria

机构 * School of Computer Science, Ariel University（计算机科学学院，阿维夫大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 9 figures

Towards Generalizable Human Activity Recognition: A Survey

Yize Cai, Baoshen Guo, Flora Salim, Zhiqing Hong

机构 * Hong Kong University of Science \& Technology (Guangzhou) China ； SMART, Massachusetts Institute of Technology Singapore ； University of New South Wales (UNSW) Australia ； Hong Kong University of Science \& Technology (Guangzhou) ； SMART, Massachusetts Institute of Technology ； University of New South Wales (UNSW) 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Evaluation of Finetuned LLMs in AMR Parsing

Shu Han Ho

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 27 pages, 32 figures

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, Dieuwke Hupkes

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Meta 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments https://aclanthology.org/2025.gem-1.33/

Journal ref Proceedings of the Fourth Workshop on Generation Evaluation and Metrics GEM2 2025 pages 404 to 430; July 31 August 1 2025; 2025 Association for Computational Linguistics

CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs

Hongtao Liu, Zhicheng Du, Zihe Wang, Weiran Shen

机构 * Gaoling School of Artificial Intelligence（北京语言大学人工智能学院） ； Renmin University of China（中国人民大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities

Noga Ben Yoash, Meni Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Lemberg, Eitam Sheetrit

机构 * Microsoft Industry AI（微软产业人工智能） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Benchmark available at: https://huggingface.co/datasets/nogabenyoash/SecQue

Journal ref n Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics, Association for Computational Linguistics (2025) https://aclanthology.org/2025.gem-1.16/

Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications

Haolin Jin, Huaming Chen

机构 * University of Sydney（悉尼大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI

Comments Accepted to the NIER track of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE 2025)

EvalAgent: Discovering Implicit Evaluation Criteria from the Web

Manya Wadhwa, Zayne Sprague, Chaitanya Malaviya, Philippe Laban, Junyi Jessy Li, Greg Durrett

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Published at COLM 2025