语言大模型 / LLM

GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics

Arthur Cho

机构 * Memoirji LLC（Memiorji公司） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 14 pages (incl. arXiv cover), 1 table, code & dataset links inside. Open-source implementation available on PyPI (grandjury package) and GitHub. Dataset available on Hugging Face under CC-BY-4.0 license

CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Tae Soo Kim, Yoonjoo Lee, Yoonah Park, Jiho Kim, Young-Ho Kim, Juho Kim

机构 * KAIST（韩国科学技术院） ； Seoul National University（首尔国立大学） ； Calvin University（凯尔文大学） ； NAVER AI LAB（NAVER AI实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to COLM 2025. Project Website: https://cupid.kixlab.org/

Evaluating the Use of LLMs for Documentation to Code Traceability

Ebube Alor, SayedHassan Khatoonabadi, Emad Shihab

机构 * Data-driven Analysis of Software (DAS) Lab Department of Computer Science \& Software Engineering Concordia University Montréal QC Canada ； Concordia University 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers

Yanzheng Xiang, Hanqi Yan, Shuyin Ouyang, Lin Gui, Yulan He

机构 * King’s College London（伦敦国王学院） ； The Alan Turing Institute（艾伦·图灵研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

RankArena: A Unified Platform for Evaluating Retrieval, Reranking and RAG with Human and LLM Feedback

Abdelrahman Abdallah, Mahmoud Abdalla, Bhawna Piryani, Jamshid Mozafari, Mohammed Ali, Adam Jatowt

专题命中 评测与基准 ：LLM(title,abstract)

Comments Accept at CIKM 2025

Meta-RAG on Large Codebases Using Code Summarization

Vali Tawosi, Salwa Alamir, Xiaomo Liu, Manuela Veloso

机构 * JP Morgan AI Research, UK（英国JPMorgan人工智能研究） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives

Wei Zeng, Hengshu Zhu, Chuan Qin, Han Wu, Yihang Cheng, Sirui Zhang, Xiaowei Jin, Yinuo Shen, Zhenxing Wang, Feimin Zhong, Hui Xiong

机构 * Business School, Hunan University（湖南大学商学院） ； Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心） ； University of Chinese Academy of Sciences（中国科学院大学） ； School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院） ； School of Business, Hunan University（湖南大学商学院） ； Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能研究所） ； Department of Computer Science and Engineering, The Hong Kong University of Science and Technology Hong Kong SAR（香港科技大学（香港）计算机科学与工程学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Tell Me Who Your Students Are: GPT Can Generate Valid Multiple-Choice Questions When Students' (Mis)Understanding Is Hinted

Machi Shimmei, Masaki Uto, Yuichiroh Matsubayashi, Kentaro Inui, Aditi Mallavarapu, Noboru Matsuda

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments This is a pre-print version of a paper to appear in AIED2025. The camera-ready version is available at https://link.springer.com/chapter/10.1007/978-3-031-99264-3_16

PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages

Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, Maarten Sap

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to COLM 2025 Main Conference

Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey

Junqiao Wang, Zeng Zhang, Yangfan He, Zihao Zhang, Xinyuan Song, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Xin Yi, Zhongwei Wan, Xinhang Yuan, Zijun Wang, Kuan Lu, Menghao Huo, Tang Jingqun, Guangwu Qian, Keqin Li, Qiuwu Chen, Lewei He

机构 * SCU（四川大学） ； UMN（明尼苏达大学） ； SCNU（华南师范大学） ； UofT（多伦多大学） ； Henan Runtai Tech（河南润泰科技） ； AIGCode ； OSU（俄亥俄州立大学） ； NJU（南京大学） ； Cornell（康奈尔大学） ； WashU（华盛顿大学） ； AMA University（阿曼大学） ； Emory（埃默里大学） ； ByteDance（字节跳动） ； UCSC（加州大学圣塔克鲁兹分校） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program

Meryem Yilmaz Soylu, Adrian Gallard, Jeonghyun Lee, Gayane Grigoryan, Rushil Desai, Stephen Harmon

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Auto-Eval Judge: Towards a General Agentic Framework for Task Completion Evaluation

Roshita Bhonsle, Rishav Dutta, Sneha Vavilapalli, Harsh Seth, Abubakarr Jaye, Yapei Chang, Mukund Rungta, Emmanuel Aboah Boateng, Sadid Hasan, Ehi Nosakhare, Soundar Srinivasan

机构 * University of Massachusetts Amherst（马萨诸塞大学阿姆赫斯特分校） ； Microsoft Corporation（微软公司） ； University of Maryland（马里兰大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；foundation model(abstract)；分类 cs.AI

The Term 'Agent' Has Been Diluted Beyond Utility and Requires Redefinition

Brinnae Bent

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to AIES 2025

A Metric for MLLM Alignment in Large-scale Recommendation

Yubin Zhang, Yanhua Huang, Haiming Xu, Mingliang Qi, Chang Wang, Jiarui Jin, Xiangyuan Ren, Xiaodan Wang, Ruiwen Xu

机构 * Xiaohongshu Inc.（小红书公司） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Pre-print.Under Review

Which Questions Improve Learning the Most? Utility Estimation of Questions with LM-based Simulations

Dong-Ho Lee, Hyundong Cho, Jonathan May, Jay Pujara

专题命中 评测与基准 ：language model(abstract)；prompting(abstract)；分类 cs.CL

Comments 17 pages, 5 figures, 6 tables

AI vs. Human Moderators: A Comparative Evaluation of Multimodal LLMs in Content Moderation for Brand Safety

Adi Levi, Or Levi, Sardhendu Mishra, Jonathan Morra

机构 * Zefr Inc（Zefr公司） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments Accepted to the Computer Vision in Advertising and Marketing (CVAM) workshop at ICCV 2025

CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation

Hamza Kalisch, Fabian Hörst, Jens Kleesiek, Ken Herrmann, Constantin Seibold

机构 * Institute for AI in Medicine (IKIM), University Hospital Essen (AöR)（人工智能医学研究所（IKIM）、埃森大学医院（AöR）） ； Department of Nuclear Medicine, University Hospital Essen (AöR)（核医学系、埃森大学医院（AöR）） ； Department of Physics, TU Dortmund（物理系、多特蒙德技术大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?

Parth Thakkar, Ankush Agarwal, Prasad Kasu, Pulkit Bansal, Chaitanya Devaguptapu

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments Accepted at ACL 2025 in the main track

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics 2025

Overview of Automatic Speech Analysis and Technologies for Neurodegenerative Disorders: Diagnosis and Assistive Applications

Shakeel A. Sheikh, Md. Sahidullah, Ina Kodrasi

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments Published in IEEE Journal of Selected Topics in Signal Processing

Journal ref https://ieeexplore.ieee.org/abstract/document/11086511/

ALScope: A Unified Toolkit for Deep Active Learning

Chenkai Wu, Yuanyuan Qi, Xiaohao Yang, Jueqing Lu, Gang Liu, Wray Buntine, Lan Du

专题命中 评测与基准 ：prompting(abstract)；分类 cs.LG

Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets

Lorenz Brehme, Thomas Ströhle, Ruth Breu

机构 * Department of Computer Science University of Innsbruck（计算机科学系因斯布鲁克大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.AI

Comments 8 Pages. This paper has been accepted for presentation at the IEEE Swiss Conference on Data Science (SDS25)

Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Action

Tongyoung Kim, Jeongeun Lee, Soojin Yoon, Sunghwan Kim, Dongha Lee

机构 * Yonsei University（延世大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.AI

EgoPrompt: Prompt Learning for Egocentric Action Recognition

Huaihai Lyu, Chaofan Chen, Yuheng Ji, Changsheng Xu

机构 * MAIS, Institute of Automation, Chinese Academy of Sciences Beijing China（MAIS，自动化研究所，中国科学院北京中国） ； MAIS, Institute of Automation, Chinese Academy of Sciences（MAIS，自动化研究所，中国科学院） ； Peng Cheng Laboratory Beijing China（鹏城实验室北京中国） ； Chinese Academy of Sciences（中国科学院） ； Peng Cheng Laboratory（鹏城实验室） 纠错

专题命中 评测与基准 ：prompting(abstract)

InSituTale: Enhancing Augmented Data Storytelling with Physical Objects

Kentaro Takahira, Yue Yu, Takanori Fujiwara, Ryo Suzuki, Huamin Qu

专题命中 评测与基准 ：LLM(abstract)

VoxRAG: A Step Toward Transcription-Free RAG Systems in Spoken Question Answering

Zackary Rackauckas, Julia Hirschberg

机构 * Columbia University（哥伦比亚大学） 纠错

专题命中 评测与基准 ：LLM(abstract)

Comments Accepted to ACL 2025 Workshop MAGMaR

Journal ref Proceedings of the 1st Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2025), pp. 40-46, Vienna, Austria, August 2025. Association for Computational Linguistics

TOGLL: Correct and Strong Test Oracle Generation with LLMs

Soneya Binta Hossain, Matthew Dwyer

专题命中 评测与基准 ：LLM(abstract)

Journal ref 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), 2025, 1475-1487

CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics

Kai Yin, Bo Li, Chengkai Liu, Ali Mostafavi, Xia Hu

专题命中 效率与部署 ：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Relevant source code and data is available: https://github.com/KaiYin97/CrsisLLM

Pruning Large Language Models by Identifying and Preserving Functional Networks

Yiheng Liu, Junhao Ning, Sichen Xia, Xiaohui Gao, Ning Qiang, Bao Ge, Junwei Han, Xintao Hu

专题命中 效率与部署 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, 5 figures

PRvL: Quantifying the Capabilities and Risks of Large Language Models for PII Redaction

Leon Garza, Anantaa Kotal, Aritran Piplai, Lavanya Elluri, Prajit Das, Aman Chadha

专题命中 效率与部署 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Camera Control at the Edge with Language Models for Scene Understanding

Alexiy Buynitsky, Sina Ehsani, Bhanu Pallakonda, Pragyana Mishra

机构 * Purdue University（普渡大学） 纠错

专题命中 效率与部署 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；SFT(abstract)

Comments 7 pages, 6 figures. This work was presented and published at the 11th IEEE International Conference on Control, Automation and Robotics (ICCAR) in 2025