语言大模型 / LLM

CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integration

Zixin Chen, Jiachen Wang, Yumeng Li, Haobo Li, Chuhan Shi, Rong Zhang, Huamin Qu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Journal ref ACM UIST 2025

Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset

Zhipeng Xue, Xiaoting Zhang, Zhipeng Gao, Xing Hu, Shan Gao, Xin Xia, Shanping Li

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Deep Language Geometry: Constructing a Metric Space from LLM Weights

Maksym Shamrai, Vladyslav Hamolia

机构 * Institute of Mathematics of NASU（国家科学院数学研究所） ； MacPaw 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 18 pages, accepted to RANLP 2025

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

David Heineman, Valentin Hofmann, Ian Magnusson, Yuling Gu, Noah A. Smith, Hannaneh Hajishirzi, Kyle Lo, Jesse Dodge

机构 * Allen Institute for Artificial Intelligence（艾伦人工智能研究所） ； Paul G. Allen School of Computer Science & Engineering, University of Washington（保罗·G·艾伦计算机科学与工程学院，华盛顿大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Deep Learning Model for Amyloidogenicity Prediction using a Pre-trained Protein LLM

Zohra Yagoub, Hafida Bouziane

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Towards Multimodal Social Conversations with Robots: Using Vision-Language Models

Ruben Janssens, Tony Belpaeme

机构 * Ghent University–imec（根特大学–imec） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL；foundation model(comments)

Comments Accepted at the workshop "Human - Foundation Models Interaction: A Focus On Multimodal Information" (FoMo-HRI) at IEEE RO-MAN 2025 (Camera-ready version)

MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation

Jianbo Dai, Jianqiao Lu, Yunlong Feng, Guangtao Zeng, Rongju Ruan, Ming Cheng, Dong Huang, Haochen Tan, Zhijiang Guo

机构 * University of Edinburgh（爱丁堡大学） ； The University of Hong Kong（香港大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Singapore University of Technology and Design（新加坡科技设计大学） ； South China University of Technology（华南理工大学） ； City University of Hong Kong（香港城市大学） ； University of Cambridge（剑桥大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 43 pages, dataset and code are available at https://github.com/SparksofAGI/MHPP, leaderboard can be found at https://sparksofagi.github.io/MHPP/

RUM: Rule+LLM-Based Comprehensive Assessment on Testing Skills

Yue Wang, Zhenyu Chen, Yuan Zhao, Chunrong Fang, Ziyuan Wang, Song Huang

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

TeleAntiFraud-28k: An Audio-Text Slow-Thinking Dataset for Telecom Fraud Detection

Zhiming Ma, Peidong Wang, Minhua Huang, Jingpeng Wang, Kai Wu, Xiangzhao Lv, Yachun Pang, Yin Yang, Wenjie Tang, Yuchen Kang

机构 * China Mobile Internet Company Ltd.（中国移动互联网有限公司） ； Northeastern University（东北大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Benchmark Dataset Generation and Evaluation for Excel Formula Repair with LLMs

Ananya Singha, Harshita Sahijwani, Walt Williams, Emmanuel Aboah Boateng, Nick Hausman, Miguel Di Luca, Keegan Choudhury, Chaya Binet, Vu Le, Tianwei Chen, Oryan Rokeah Chen, Sulaiman Vesal, Sadid Hasan

机构 * Microsoft（微软） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted at the KDD workshop on Evaluation and Trustworthiness of Agentic and Generative AI Models

What do Speech Foundation Models Learn? Analysis and Applications

Ankita Pasad

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.CL

Comments Ph.D. Thesis

LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data

Stephen Meisenbacher, Alexandra Klymenko, Florian Matthes

机构 * Technical University of Munich School of Computation, Information（慕尼黑技术大学计算、信息与技术学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Comments 13 pages, 3 figures, 4 tables. Accepted to HAIPS @ CCS 2025

Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challenges

Lajos Muzsai, David Imolai, András Lukács

机构 * Eötvös Loránd University, Institute of Mathematics, AI Research Group（埃奥武斯·洛尔兰大学数学学院人工智能研究组） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI

Comments 13 pages, 2 figures

Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality

Rahul Zalkikar, Kanchan Chandra

机构 * New York University（纽约大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pages 1337--1361

The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution

Elon Ezra, Ariel Weizman, Amos Azaria

机构 * School of Computer Science, Ariel University（计算机科学学院，阿维夫大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 9 figures

Towards Generalizable Human Activity Recognition: A Survey

Yize Cai, Baoshen Guo, Flora Salim, Zhiqing Hong

机构 * Hong Kong University of Science \& Technology (Guangzhou) China ； SMART, Massachusetts Institute of Technology Singapore ； University of New South Wales (UNSW) Australia ； Hong Kong University of Science \& Technology (Guangzhou) ； SMART, Massachusetts Institute of Technology ； University of New South Wales (UNSW) 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Evaluation of Finetuned LLMs in AMR Parsing

Shu Han Ho

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 27 pages, 32 figures

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, Dieuwke Hupkes

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Meta 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments https://aclanthology.org/2025.gem-1.33/

Journal ref Proceedings of the Fourth Workshop on Generation Evaluation and Metrics GEM2 2025 pages 404 to 430; July 31 August 1 2025; 2025 Association for Computational Linguistics

CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs

Hongtao Liu, Zhicheng Du, Zihe Wang, Weiran Shen

机构 * Gaoling School of Artificial Intelligence（北京语言大学人工智能学院） ； Renmin University of China（中国人民大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities

Noga Ben Yoash, Meni Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Lemberg, Eitam Sheetrit

机构 * Microsoft Industry AI（微软产业人工智能） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Benchmark available at: https://huggingface.co/datasets/nogabenyoash/SecQue

Journal ref n Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics, Association for Computational Linguistics (2025) https://aclanthology.org/2025.gem-1.16/

Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications

Haolin Jin, Huaming Chen

机构 * University of Sydney（悉尼大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI

Comments Accepted to the NIER track of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE 2025)

EvalAgent: Discovering Implicit Evaluation Criteria from the Web

Manya Wadhwa, Zayne Sprague, Chaitanya Malaviya, Philippe Laban, Junyi Jessy Li, Greg Durrett

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Published at COLM 2025

Rethinking Autonomy: Preventing Failures in AI-Driven Software Engineering

Satyam Kumar Navneet, Joydeep Chandra

机构 * Department of CSE Chandigarh University Mohali, India（计算机科学与工程系 奇纳格里大学 莫哈利，印度） ； Department of CST Tsinghua University Beijing, China（计算机科学与技术系 清华大学 北京，中国） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

A Multi-Task Evaluation of LLMs' Processing of Academic Text Input

Tianyi Li, Yu Qin, Olivia R. Liu Sheng

机构 * Department of Decisions, Operations and Technology, CUHK（决策、运营与技术系，香港中文大学） ； Department of Information Systems, Arizona State University（信息系统系，亚利桑那州立大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

iShumei-Chinchunmei at SemEval-2025 Task 4: A balanced forgetting and retention multi-task framework using effective unlearning loss

Yujian Sun, Tian Li

机构 * Shumei AI Research Institute（Shumei人工智能研究院） ； School of Computing（计算学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Journal ref Association for Computational Linguistics, Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), 1357-1369

ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images

Wenjie Liao, Jieyu Yuan, Yifang Xu, Chunle Guo, Zilong Zhang, Jihong Li, Jiachen Fu, Haotian Fan, Tao Li, Junhui Cui, Chongyi Li

机构 * Media Evaluation Lab, ByteDance Inc.（字节跳动公司媒体评估实验室） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)

UniDCF: A Foundation Model for Comprehensive Dentocraniofacial Hard Tissue Reconstruction

Chunxia Ren, Ning Zhu, Yue Lai, Gui Chen, Ruijie Wang, Yangyi Hu, Suyao Liu, Shuwen Mao, Hong Su, Yu Zhang, Li Xiao

专题命中 评测与基准 ：foundation model(title)；分类 cs.AI

Comments 23 pages, 6 figures

Beyond Modality Limitations: A Unified MLLM Approach to Automated Speaking Assessment with Effective Curriculum Learning

Yu-Hsuan Fang, Tien-Hong Lo, Yao-Ting Sung, Berlin Chen

机构 * National Taiwan Normal University（台湾国立正常大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IEEE ASRU 2025

M$^3$FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial Meeting Understanding Evaluation Dataset

Jie Zhu, Junhui Li, Yalong Wen, Xiandong Li, Lifan Guo, Feng Chen

机构 * School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院） ； Qwen DianJin Team, Alibaba Cloud Computing（阿里云量子金融团队） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL-2025

Journal ref The 63nd Annual Meeting of the Association for Computational Linguistics(ACL),2025

Doğal Dil İşlemede Tokenizasyon Standartları ve Ölçümü: Türkçe Üzerinden Büyük Dil Modellerinin Karşılaştırmalı Analizi

M. Ali Bayram, Ali Arda Fincan, Ahmet Semih Gümüş, Sercan Karakaş, Banu Diri, Savaş Yıldırım

机构 * Yıldız Technical University（亚兹德技术大学） ； Yeditepe University（耶迪特佩大学） ； The University of Chicago（芝加哥大学） ； İstanbul Bilgi University（伊斯坦布尔比尔大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments in Turkish language, Presented at the 2025 33rd Signal Processing and Communications Applications Conference (SIU), 25--28 June 2025, Şile, Istanbul, Türkiye