语言大模型 / LLM

Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?

Burak Can Kaplan, Hugo Cesar De Castro Carneiro, Stefan Wermter

机构 * Department of Informatics, University of Hamburg（信息学院，汉堡大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 4 figures

Can open source large language models be used for tumor documentation in Germany? -- An evaluation on urological doctors' notes

Stefan Lenz, Arsenij Ustjanzew, Marco Jeray, Meike Ressing, Torsten Panholzer

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 53 pages, 5 figures

Journal ref BioData Mining volume 18, Article number 48 (2025)

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

Dexuan Xu, Jieyi Wang, Zhongyan Chai, Yongzhi Cao, Hanpin Wang, Huamin Zhang, Yu Huang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 18 pages

McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models

Tian Lan, Xiangdong Su, Xu Liu, Ruirui Wang, Ke Chang, Jiang Li, Guanglai Gao

机构 * College of Computer Science, Inner Mongolia University（内蒙古大学计算机科学学院） ； National & Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian（蒙古语智能信息处理技术国家与地方联合工程研究中心） ； Inner Mongolia Key Laboratory of Multilingual Artificial Intelligence Technology（内蒙古多语种人工智能技术重点实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL2025 Findings

Journal ref In Findings of the Association for Computational Linguistics: ACL 2025, pages 6033-6056, Vienna, Austria. Association for Computational Linguistics

Large Language Models Still Exhibit Bias in Long Text

Wonje Jeung, Dongjae Jeon, Ashkan Yousefpour, Jonghyun Choi

机构 * Yonsei University（延世大学） ； Seoul National University（首尔国立大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL, code and models are available at https://github.com/WonjeJeung/LTF-TEST

Optimizing LLM-Based Multi-Agent System with Textual Feedback: A Case Study on Software Development

Ming Shen, Raphael Shu, Anurag Pratik, James Gung, Yubin Ge, Monica Sunkara, Yi Zhang

机构 * Arizona State University（亚利桑那州立大学） ； Amazon Web Services（亚马逊网络服务） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

LLaVA-RE: Binary Image-Text Relevancy Evaluation with Multimodal Large Language Model

Tao Sun, Oliver Liu, JinJin Li, Lan Ma

机构 * Stony Brook University（斯通布罗克大学） ； Amazon（亚马逊） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)

Comments Published in the First Workshop of Evaluation of Multi-Modal Generation 2025

TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution

Zhikai Zhao, Chuanbo Hua, Federico Berto, Kanghoon Lee, Zihan Ma, Jiachen Li, Jinkyoo Park

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2505.04480

Who is a Better Player: LLM against LLM

Yingjie Zhou, Jiezhang Cao, Farong Wen, Li Xu, Yanwei Jiang, Jun Jia, Ronghui Li, Xiaohong Liu, Yu Zhou, Xiongkuo Min, Jie Guo, Zicheng Zhang, Guangtao Zhai

机构 * Shanghai Jiao Tong University（上海交通大学） ； PengCheng Laboratory（鹏城实验室） ； Harvard Medical School（哈佛医学院） ； Shanghai AI Laboratory（上海人工智能实验室） ； China University of Mining and Technology（中国矿业大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Recent Advances in Speech Language Models: A Survey

Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng, Guangyan Zhang, Qichao Wang, Yiwen Guo, Irwin King

机构 * Department of Computer Science and Engineering, The Chinese University of Hong Kong（计算机科学与工程系，香港中文大学） ； LIGHTSPEED STUDIOS ； National University of Singapore（新加坡国立大学） ； Tencent（腾讯） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL

Comments The reduced version of this paper has been accepted at ACL 2025

Tool Graph Retriever: Exploring Dependency Graph-based Tool Retrieval for Large Language Models

Linfeng Gao, Yaoxiang Wang, Minlong Peng, Jialong Tang, Yuzhe Shang, Mingming Sun, Jinsong Su

机构 * School of Informatics, Xiamen University（厦门大学信息学院） ； Baidu（百度） ； Alibaba（阿里巴巴） 纠错

专题命中 评测与基准 ：large language model(title)；language model(title)；分类 cs.AI

The SMeL Test: A simple benchmark for media literacy in language models

Gustaf Ahdritz, Anat Kleiman

机构 * Kempner Institute, Harvard University（凯普纳研究所，哈佛大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations

Brandon Jaipersaud, David Krueger, Ekdeep Singh Lubana

机构 * Mila ； University of Montreal（蒙特利尔大学） ； Harvard University（哈佛大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

TASE: Token Awareness and Structured Evaluation for Multilingual Language Models

Chenzhuo Zhao, Xinda Wang, Yue Huang, Junting Lu, Ziqian Liu

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations

Julia Kharchenko, Tanya Roosta, Aman Chadha, Chirag Shah

机构 * University of Washington（华盛顿大学） ； UC Berkeley（伯克利大学） ； Stanford University（斯坦福大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Augmented Question-guided Retrieval (AQgR) of Indian Case Law with LLM, RAG, and Structured Summaries

Vishnuprabha V, Daleesha M Viswanathan, Rajesh R, Aneesh V Pillai

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

Enriching language models with graph-based context information to better understand textual data

Albert Roethel, Maria Ganzha, Anna Wróblewska

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 5 figures

Adapting Vision-Language Models Without Labels: A Comprehensive Survey

Hao Dong, Lijun Sheng, Jian Liang, Ran He, Eleni Chatzi, Olga Fink

机构 * ETH Zürich（苏黎世联邦理工学院） ； University of Science and Technology of China（中国科学技术大学） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； EPFL（苏黎世联邦理工学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Discussions, comments, and questions are welcome in \url{https://github.com/tim-learn/Awesome-LabelFree-VLMs}

Can Vision Language Models Understand Mimed Actions?

Hyundong Cho, Spencer Lin, Tejas Srinivasan, Michael Saxon, Deuksin Kwon, Natali T. Chavez, Jonathan May

机构 * Information Sciences Institute（信息科学研究所） ； Institute for Creative Technologies（创意技术研究所） ； Department of Computer Science（计算机科学系） ； University of Southern California（南加州大学） ； University of California, Santa Barbara（加州大学圣巴巴拉分校） ； Aristotle University of Thessaloniki（希腊雅典纳大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Findings

ArXivBench: When You Should Avoid Using ChatGPT for Academic Writing

Ning Li, Jingran Zhang, Justin Cui

机构 * University of California, Los Angeles（加州大学洛杉矶分校） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

VS-LLM: Visual-Semantic Depression Assessment based on LLM for Drawing Projection Test

Meiqi Wu, Yaxuan Kang, Xuchen Li, Shiyu Hu, Xiaotang Chen, Yunfeng Kang, Weiqiang Wang, Kaiqi Huang

机构 * School of Computer Science and Technology, University of Chinese Academy of Sciences（计算机科学与技术学院，中国科学院大学） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学） ； CAS Center for Excellence in Brain Science and Intelligence Technology（脑科学与智能技术卓越中心） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI

Evaluating LLM-Generated Q&A Test: a Student-Centered Study

Anna Wróblewska, Bartosz Grabek, Jakub Świstak, Daniel Dan

机构 * Warsaw University of Technology Faculty of Mathematics and Information Science（华沙技术大学数学与信息科学学院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Comments accepted to AIED 2025

GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics

Arthur Cho

机构 * Memoirji LLC（Memiorji公司） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 14 pages (incl. arXiv cover), 1 table, code & dataset links inside. Open-source implementation available on PyPI (grandjury package) and GitHub. Dataset available on Hugging Face under CC-BY-4.0 license

CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Tae Soo Kim, Yoonjoo Lee, Yoonah Park, Jiho Kim, Young-Ho Kim, Juho Kim

机构 * KAIST（韩国科学技术院） ； Seoul National University（首尔国立大学） ； Calvin University（凯尔文大学） ； NAVER AI LAB（NAVER AI实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to COLM 2025. Project Website: https://cupid.kixlab.org/

Evaluating the Use of LLMs for Documentation to Code Traceability

Ebube Alor, SayedHassan Khatoonabadi, Emad Shihab

机构 * Data-driven Analysis of Software (DAS) Lab Department of Computer Science \& Software Engineering Concordia University Montréal QC Canada ； Concordia University 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers

Yanzheng Xiang, Hanqi Yan, Shuyin Ouyang, Lin Gui, Yulan He

机构 * King’s College London（伦敦国王学院） ； The Alan Turing Institute（艾伦·图灵研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

RankArena: A Unified Platform for Evaluating Retrieval, Reranking and RAG with Human and LLM Feedback

Abdelrahman Abdallah, Mahmoud Abdalla, Bhawna Piryani, Jamshid Mozafari, Mohammed Ali, Adam Jatowt

专题命中 评测与基准 ：LLM(title,abstract)

Comments Accept at CIKM 2025

Meta-RAG on Large Codebases Using Code Summarization

Vali Tawosi, Salwa Alamir, Xiaomo Liu, Manuela Veloso

机构 * JP Morgan AI Research, UK（英国JPMorgan人工智能研究） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives

Wei Zeng, Hengshu Zhu, Chuan Qin, Han Wu, Yihang Cheng, Sirui Zhang, Xiaowei Jin, Yinuo Shen, Zhenxing Wang, Feimin Zhong, Hui Xiong

机构 * Business School, Hunan University（湖南大学商学院） ； Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心） ； University of Chinese Academy of Sciences（中国科学院大学） ； School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院） ； School of Business, Hunan University（湖南大学商学院） ； Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能研究所） ； Department of Computer Science and Engineering, The Hong Kong University of Science and Technology Hong Kong SAR（香港科技大学（香港）计算机科学与工程学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Tell Me Who Your Students Are: GPT Can Generate Valid Multiple-Choice Questions When Students' (Mis)Understanding Is Hinted

Machi Shimmei, Masaki Uto, Yuichiroh Matsubayashi, Kentaro Inui, Aditi Mallavarapu, Noboru Matsuda

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments This is a pre-print version of a paper to appear in AIED2025. The camera-ready version is available at https://link.springer.com/chapter/10.1007/978-3-031-99264-3_16