语言大模型 / LLM

R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation

Futian Wang, Yuhan Qiao, Xiao Wang, Fuling Wang, Yuxiang Zhang, Dengdi Sun

机构 * School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Pay What LLM Wants: Can LLM Simulate Economics Experiment with 522 Real-human Persona?

Junhyuk Choi, Hyeonchu Park, Haemin Lee, Hyebeen Shin, Hyun Joung Jin, Bugeun Kim

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Preprint

Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework

Zikun Cui, Tianyi Huang, Chia-En Chiang, Cuiqianhe Du

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

The influence of persona and conversational task on social interactions with a LLM-controlled embodied conversational agent

Leon O. H. Kroczek, Alexander May, Selina Hettenkofer, Andreas Ruider, Bernd Ludwig, Andreas Mühlberger

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 11 pages, 5 figures

VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs

Zixuan Gu, Qiufeng Fan, Long Sun, Yang Liu, Xiaojun Ye

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 12 pages, 10 figures, published in KDD2025

Journal ref In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD'25), August 3-7, 2025, Toronto, ON, Canada. ACM, New York, NY, USA, 12 pages

Automated Validation of LLM-based Evaluators for Software Engineering Artifacts

Ora Nova Fandina, Eitan Farchi, Shmulik Froimovich, Rami Katan, Alice Podolsky, Orna Raz, Avi Ziv

机构 * IBM Research（IBM研究院） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Enhancing Spectral Graph Neural Networks with LLM-Predicted Homophily

Kangkang Lu, Yanhua Yu, Zhiyong Huang, Tat-Seng Chua

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark

Goeric Huybrechts, Srikanth Ronanki, Sai Muralidhar Jayanthi, Jack Fitzgerald, Srinivasan Veeravanallur

机构 * Amazon AGI（亚马逊人工智能研究院） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes

Shahed Masoudian, Gustavo Escobedo, Hannah Strauss, Markus Schedl

机构 * Johannes Kepler University (JKU)（约翰内斯·开普勒大学） ； Linz Institute of Technology (LIT)（林茨技术研究所） ； University of Innsbruck（因斯布鲁克大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Under Review

Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness Criteria

Michael S. Yao, Allison Chae, Charles E. Kahn, Walter R. Witschey, James C. Gee, Hersh Sagreiya, Osbert Bastani

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments 15 pages main text, 4 figures, 1 table

Journal ref Commun Med 5, 332 (2025)

Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework

Jialin Li, Jinzhe Li, Gengxu Li, Yi Chang, Yuan Wu

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Models

Wenjie Luo, Ruocheng Li, Shanshan Zhu, Julian Perry

机构 * Fujian University of Technology（福建工程大学） ； Delta University for Science and Technology（科技大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

CHIRP: A Fine-Grained Benchmark for Open-Ended Response Evaluation in Vision-Language Models

Alexis Roger, Prateek Humane, Daniel Z. Kaplan, Kshitij Gupta, Qi Sun, George Adamopoulos, Jonathan Siu Chi Lim, Quentin Anthony, Edwin Fennell, Irina Rish

机构 * Mila - Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； Tokyo Institute of Technology（东京技术大学） ； McGill University（麦吉尔大学） ； EleutherAI ； University College London（伦敦大学学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI

VQA support to Arabic Language Learning Educational Tool

Khaled Bachir Delassi, Lakhdar Zeggane, Hadda Cherroun, Abdelhamid Haouhat, Kaoutar Bouzouad

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；prompting(abstract)

A System Model Generation Benchmark from Natural Language Requirements

Dongming Jin, Zhi Jin, Linyu Li, Zheng Fang, Jia Li, Xiaohong Chen

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 16 pages, 14 figures

Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation

Radhika Dua, Young Joon, Kwon, Siddhant Dogra, Daniel Freedman, Diana Ruan, Motaz Nashawaty, Danielle Rigau, Daniel Alexander Alber, Kang Zhang, Kyunghyun Cho, Eric Karl Oermann

机构 * New York University（纽约大学） ； NYU Langone Health（纽约大学兰格医学中心） ； Genentech（基因泰克） ； NYU Grossman School of Medicine（纽约大学格罗斯曼医学院） ； Wenzhou Medical University（温州医科大学） ； Macau University of Science and Technology（澳门科学大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

VisuCraft: Enhancing Large Vision-Language Models for Complex Visual-Guided Creative Content Generation via Structured Information Extraction

Rongxin Jiang, Robert Long, Chenghao Gu, Mingrui Yan

机构 * Heilongjiang University of Science and Technology（黑龙江科技大学） ； University of Padua（帕多瓦大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications

Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, Chunfeng Song

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； The Chinese University of Hong Kong（香港中文大学） ； Tsinghua University（清华大学） ； North China Electric Power University（华北电力大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.LG

A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models

Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiao-yong Wei, Shanru Lin, Hui Liu, Philip S. Yu, Qing Li

机构 * The Hong Kong Polytechnic University（香港理工大学） ； City University of Hong Kong（香港城市大学） ； Michigan State University（密歇根州立大学） ； University of Illinois at Chicago（伊利诺伊大学芝加哥分校） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；分类 cs.AI

Comments This is the long version of the corresponding survey paper accepted by KDD 2025. The tutorial and corresponding slides are available at https://biglemon-ning.github.io/WebAgents/

Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics

Wenrui Xu, Dalin Lyu, Weihang Wang, Jie Feng, Chen Gao, Yong Li

机构 * School of Architecture, Tsinghua University（清华大学建筑学院） ； Department of Electronic Engineering, Tsinghua University（清华大学电子工程系） ； BNRist, Tsinghua University（清华大学BNRist） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics. Volume 1: Long Papers (2025) 11571-11590

A Comprehensive Evaluation of Semantic Relation Knowledge of Pretrained Language Models and Humans

Zhihan Cao, Hiroaki Yamada, Simone Teufel, Takenobu Tokunaga

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Journal ref Language Resources & Evaluation (2025)

M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs

Junwoo Ha, Hyunjun Kim, Sangyoon Yu, Haon Park, Ashkan Yousefpour, Yuna Park, Suhyun Kim

机构 * AIM Intelligence（AIM智能研究所） ； University of Seoul（首尔大学） ； Korea Advanced Institute of Science and Technology（韩国科学技术院） ； Seoul National University（首尔国立大学） ； Yonsei University（延世大学） ； Korea Institute of Science and Technology（韩国科学技术院） ； Kyung Hee University（庆熙大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2025 (Main Track). Camera-ready version

Journal ref Proc. ACL 2025 (Vol. 1: Long Papers), pp. 16489-16507, Vienna, Austria, 2025

GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking

Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, Simone Paolo Ponzetto

专题命中 评测与基准 ：LLM(title,abstract)

Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies

Yi Ma, Hongyao Tang, Chenjun Xiao, Yaodong Yang, Wei Wei, Jianye Hao, Jiye Liang

机构 * School of Computer and Information Technology, Shanxi University（山西大学计算机与信息技术学院） ； College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院） ； School of Data Science, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）数据科学学院） ； Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

PentestJudge: Judging Agent Behavior Against Operational Requirements

Shane Caldwell, Max Harley, Michael Kouremetis, Vincent Abruzzo, Will Pearce

机构 * dreadnode 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 18 pages, 5 figures, 3 tables

GEMA-Score: Granular Explainable Multi-Agent Scoring Framework for Radiology Report Evaluation

Zhenxuan Zhang, Kinhei Lee, Peiyuan Jing, Weihang Deng, Huichi Zhou, Zihao Jin, Jiahao Huang, Zhifan Gao, Dominic C Marshall, Yingying Fang, Guang Yang

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

SlideAudit: A Dataset and Taxonomy for Automated Evaluation of Presentation Slides

Zhuohao Jerry Zhang, Ruiqi Chen, Mingyuan Zhong, Jacob O. Wobbrock

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)

Comments UIST 2025

Proof2Hybrid: Automatic Mathematical Benchmark Synthesis for Proof-Centric Problems

Yebo Peng, Zixiang Liu, Yaoming Li, Zhizhuo Yang, Xinye Xu, Bowen Ye, Weijun Yuan, Zihan Wang, Tong Yang

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI4Research: A Survey of Artificial Intelligence for Scientific Research

Qiguang Chen, Mingda Yang, Libo Qin, Jinhao Liu, Zheng Yan, Jiannan Guan, Dengyun Peng, Yiyan Ji, Hanjing Li, Mengkang Hu, Yimeng Zhang, Yihao Liang, Yuhang Zhou, Jiaqi Wang, Zhi Chen, Wanxiang Che

机构 * LARG, Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology（哈尔滨工业大学大规模学习与推理中心，社会计算与交互机器人研究所以） ； School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院） ； The University of Hong Kong（香港大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Princeton University（普林斯顿大学） ； Fudan University（复旦大学） ； Chinese University of Hong Kong（香港中文大学） ； ByteDance Seed (China)（字节跳动种子（中国）） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Preprint, Paper list is available at https://github.com/LightChen233/Awesome-AI4Research

Can LLMs Generate High-Quality Task-Specific Conversations?

Shengqi Li, Amarnath Gupta

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI