语言大模型 / LLM

IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models

Yiming Gao, Bin Wang, Chengwei Wei, Shuo Sun, AiTi Aw

机构 * Nanyang Technological University (NTU)（南洋理工大学） ； MiroMind（米罗Mind） ； Institute for Infocomm Research (I 2 R)（信息与通信研究院） ； A*STAR（科技研究局） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Link: https://github.com/AudioLLMs/AudioBench/tree/main/IFEval-Audio

Large Language Models Are Unreliable for Cyber Threat Intelligence

Emanuele Mezzi, Fabio Massacci, Katja Tuma

机构 * Springer Nature Switzerland（斯普林格·自然瑞士） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

OKBench: Democratizing LLM Evaluation with Fully Automated, On-Demand, Open Knowledge Benchmarking

Yanhong Li, Tianyang Xu, Kenan Tang, Karen Livescu, David McAllester, Jiawei Zhou

机构 * University of Chicago（芝加哥大学） ； TTI-Chicago（芝加哥技术研究所） ； UC Santa Barbara（圣巴巴拉大学） ； Stony Brook University（石溪大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Evaluating Deep Unlearning in Large Language Models

Ruihan Wu, Chhavi Yadav, Russ Salakhutdinov, Kamalika Chaudhuri

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；分类 cs.CL

MTQ-Eval: Multilingual Text Quality Evaluation for Language Models

Rhitabrat Pokharel, Ameeta Agrawal

机构 * Department of Computer Science（计算机科学系） ； Portland State University（波特兰州立大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IJCNLP-AACL 2025 Findings

ElicitationGPT: Text Elicitation Mechanisms via Language Models

Yifan Wu, Jason Hartline

机构 * Microsoft Research（微软研究院） ； Northwestern University（西北大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Detecting Emotional Dynamic Trajectories: An Evaluation Framework for Emotional Support in Language Models

Zhouxing Tan, Ruochong Xiong, Yulong Wan, Jinlong Ma, Hanlin Xue, Qichun Deng, Haifeng Jing, Zhengtong Zhang, Depei Liu, Shiyuan Luo, Junfei Liu

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Models

Patrick Haller, Fabio Barth, Jonas Golde, Georg Rehm, Alan Akbik

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments 8 pages, 11 tables and figures

AutoSynth: Automated Workflow Optimization for High-Quality Synthetic Dataset Generation via Monte Carlo Tree Search

Shuzhen Bi, Chang Song, Siyu Song, Jinze Lv, Jian Chen, Xinyun Wang, Aimin Zhou, Hao Hao

机构 * Shanghai Innavation Institute（上海创新研究院） ； Shanghai Institute of AI for Education（上海人工智能教育研究院） ； Department of Education Information Technology（教育信息技术部） ； School of Computer Science and Technology（计算机科学与技术学院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

mmJEE-Eval: A Bilingual Multimodal Benchmark for Evaluating Scientific Reasoning in Vision-Language Models

Arka Mukherjee, Shreya Ghosh

机构 * Kalinga Institute of Industrial Technology (KIIT)（喀里亚理工学院） ； Indian Institute of Technology (IIT), Bhubaneswar（印度理工学院（班加罗尔）） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to IJCNLP-AACL Findings 2025

IndoPref: A Multi-Domain Pairwise Preference Dataset for Indonesian

Vanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, Genta Indra Winata

机构 * Institut Teknologi Bandung（技术科技大学） ； Stanford University（斯坦福大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by IJCNLP-AACL 2025

PET2Rep: Towards Vision-Language Model-Drived Automated Radiology Report Generation for Positron Emission Tomography

Yichi Zhang, Wenbo Zhang, Zehui Ling, Gang Feng, Sisi Peng, Deshu Chen, Yuchen Liu, Hongwei Zhang, Shuqi Wang, Lanlan Li, Limei Han, Yuan Cheng, Zixin Hu, Yuan Qi, Le Xue

专题命中 评测与基准 ：language model(title,abstract)

Comments Accepted by AAAI 2026

CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Models

Xiao An, Jiaxing Sun, Zihan Gui, Wei He

机构 * State Key Lab. LIESMARS, Wuhan University（武汉大学遥感信息与地学实验室国家重点实验室） 纠错

专题命中 评测与基准 ：language model(title,abstract)

Comments Accepted by NeurIPS 2025 Track on Datasets and Benchmarks

UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets

Adrien Meyer, Aditya Murali, Farahdiba Zarin, Didier Mutter, Nicolas Padoy

机构 * University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France（斯特拉斯堡大学，法国国家科学研究中心，法国国家卫生研究院，ICube，UMR7357，斯特拉斯堡，法国） ； IHU-Strasbourg, Institute of Image-Guided Surgery, Strasbourg, France（斯特拉斯堡IHU，影像引导手术研究所，斯特拉斯堡，法国） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)

Comments 7 pages, 3 figures, 3 tables

LiteraryTaste: A Preference Dataset for Creative Writing Personalization

John Joon Young Chung, Vishakh Padmakumar, Melissa Roemmele, Yi Wang, Yuqian Sun, Tiffany Wang, Shm Garanganao Almeda, Brett A. Halperin, Yuwen Lu, Max Kreminski

机构 * Stanford University（斯坦福大学） ； University of Washington（华盛顿大学） ； University of Notre Dame（圣母大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Sure! Here's a short and concise title for your paper: "Contamination in Generated Text Detection Benchmarks"

Philipp Dingfelder, Christian Riess

机构 * IT Security Infrastructures Lab, FAU Erlangen-Nürnberg（FAU埃尔兰根-纽伦堡大学信息安全部署实验室） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments published at CSCML 2025

Context is Enough: Empirical Validation of $\textit{Sequentiality}$ on Essays

Amal Sunny, Advay Gupta, Vishnu Sreekumar

机构 * IIIT-Hyderabad（IIIT-海得拉巴） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages, 2 figures

BioVerge: A Comprehensive Benchmark and Study of Self-Evaluating Agents for Biomedical Hypothesis Generation

Fuyi Yang, Chenchen Ye, Mingyu Derek Ma, Yijia Xiao, Matthew Yang, Wei Wang

机构 * University of California, Los Angeles（加州大学洛杉矶分校） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

MADD: Multi-Agent Drug Discovery Orchestra

Gleb V. Solovev, Alina B. Zhidkovskaya, Anastasia Orlova, Nina Gubina, Anastasia Vepreva, Rodion Golovinskii, Ilya Tonkii, Ivan Dubrovsky, Ivan Gurev, Dmitry Gilemkhanov, Denis Chistiakov, Timur A. Aliev, Ivan Poddiakov, Galina Zubkova, Ekaterina V. Skorb, Vladimir Vinogradov, Alexander Boukhanovsky, Nikolay Nikitin, Andrei Dmitrenko, Anna Kalyuzhnaya, Andrey Savchenko

机构 * ITMO University（ITMO大学） ； Sber AI Lab（Sber AI实验室） ； D ONE AG ； HSE University（高等经济大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments EMNLP2025 accepted paper, Findings 2025

Leveraging Small LLMs for Argument Mining in Education: Argument Component Identification, Classification, and Assessment

Lucile Favero, Juan Antonio Pérez-Ortiz, Tanja Käser, Nuria Oliver

机构 * ELLIS Alicante, Spain（阿利坎特ELLIS研究所，西班牙） ； Universitat d’Alacant, Spain（阿利坎特大学，西班牙） ； École Polytechnique Fédérale de Lausanne, EPFL, Switzerland（洛桑联邦理工学院，瑞士） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

The Feasibility of Topic-Based Watermarking on Academic Peer Reviews

Alexander Nemecek, Yuzhou Jiang, Erman Ayday

机构 * Case Western Reserve University（凯斯西储大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted at AACL 25 Findings

TempoQL: A Readable, Precise, and Portable Query System for Electronic Health Record Data

Ziyong Ma, Richard D. Boyce, Adam Perer, Venkatesh Sivaraman

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted as a Proceedings paper at Machine Learning for Health (ML4H) 2025

BronchOpt : Vision-Based Pose Optimization with Fine-Tuned Foundation Models for Accurate Bronchoscopy Navigation

Hongchao Shu, Roger D. Soberanis-Mukul, Jiru Xu, Hao Ding, Morgan Ringel, Mali Shen, Saif Iftekar Sayed, Hedyeh Rafii-Tari, Mathias Unberath

机构 * Johnson & Johnson MedTech（强生医疗科技） 纠错

专题命中 评测与基准 ：foundation model(title)；分类 cs.AI

BIG5-TPoT: Predicting BIG Five Personality Traits, Facets, and Items Through Targeted Preselection of Texts

Triet M. Le, Arjun Chandra, C. Anton Rytting, Valerie P. Karuzis, Vladimir Rife, William A. Simpson

机构 * The University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)（马里兰大学应用研究实验室（情报与安全实验室）） ； The University of Maryland (UMD)（马里兰大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

A Hybrid Search for Complex Table Question Answering in Securities Report

Daiki Shirafuji, Koji Tanaka, Tatsuhiko Saito

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to IIAI AAI 2025

Too Good to be Bad: On the Failure of LLMs to Role-Play Villains

Zihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen, Qu Yang, Mengru Wang, Fanghua Ye, Ying Shen, Zhaopeng Tu, Xiaolong Li, Linus

机构 * Tencent（腾讯） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding

Haitao Li, Ziyu Li, Yiheng Mao, Ziyi Liu, Zhoujian Sun, Zhengxing Huang

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments AAAI 2026

What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models

Keyon Vafa, Sarah Bentley, Jon Kleinberg, Sendhil Mullainathan

机构 * Harvard University（哈佛大学） ； MIT（麻省理工学院） ； Cornell University（康奈尔大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Multi-scale Cascaded Foundation Model for Whole-body Organs-at-risk Segmentation

Rui Hao, Dayu Tan, Qiankun Li, Chunhou Zheng, Weimin Zhong, Zhigang Zeng

机构 * School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学） ； Institute of Artificial Intelligence, Huazhong University of Science and Technology（人工智能研究院，华中科技大学） ； Hubei Key Laboratory of Brain-Inspired Intelligent Systems, Huazhong University of Science and Technology（湖北省脑启发智能系统重点实验室，华中科技大学） ； Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education（图像处理与智能控制重点实验室（华中科技大学），教育部） ； Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University（智能计算与信号处理重点实验室（安徽大学），教育部） ； College of Computing and Data Science (CCDS), Nanyang Technological University（计算与数据科学学院（CCDS），南洋理工大学） ； East China University of Science and Technology（东华大学） 纠错

专题命中 评测与基准 ：foundation model(title)

MCAD: Multimodal Context-Aware Audio Description Generation For Soccer

Lipisha Chaudhary, Trisha Mittal, Subhadra Gopalakrishnan, Ifeoma Nwogu, Jaclyn Pytlarz

机构 * University at Buffalo, SUNY（布法罗大学，SUNY） ； Dolby Laboratories Inc.（杜比实验室公司） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.LG