语言大模型 / LLM - arXivDaily 专题

2506.08375 2025-09-17 cs.CL 89%

EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models

Tao Zou, Xinghua Zhang, Haiyang Yu, Minzheng Wang, Fei Huang, Yongbin Li

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2025

2505.15805 2025-09-17 cs.CL 89%

Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering

Hwan Chang, Yumin Kim, Yonghyun Jun, Hwanhee Lee

机构 * Chung-Ang University（Chung-Ang 大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments EMNLP 2025 (Main Conference)

2509.12973 2025-09-17 cs.SE 89%

Evaluating Large Language Models for Code Translation: Effects of Prompt Language and Prompt Design

Aamer Aljagthami, Mohammed Banabila, Musab Alshehri, Mohammed Kabini, Mohammad D. Alahmadi

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2509.06164 2025-09-17 cs.CL cs.AI cs.HC cs.LG 89%

Benchmarking Gender and Political Bias in Large Language Models

Jinrui Yang, Xudong Han, Timothy Baldwin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2509.12602 2025-09-17 cs.AI cs.CL 88%

DaSAThco: Data-Aware SAT Heuristics Combinations Optimization via Large Language Models

Minyu Chen, Guoqiang Li

机构 * Minyu Chen, Guoqiang Li（作者）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages

2505.19345 2025-09-17 cs.CL cs.AI 86%

PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims

Yongmin Yoo, Qiongkai Xu, Longbing Cao

机构 * Frontier AI Research Centre, Macquarie University School of Computing, FSE, Macquarie University（前沿人工智能研究中心、麦考瑞大学计算机学院、FSE、麦考瑞大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2502.18530 2025-09-17 cs.CV cs.LG 85%

IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Experts

Eric Xue, Ke Chen, Zeyi Huang, Yuyang Ji, Haohan Wang

机构 * University of Toronto（多伦多大学）； University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； University of Wisconsin-Madison（威斯康星大学麦迪逊分校）； New York University（纽约大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2509.12678 2025-09-17 cs.LG cs.AI 84%

Instance-level Randomization: Toward More Stable LLM Evaluations

Yiyang Li, Yonghuang Wu, Ying Luo, Liangtai Sun, Zishu Qin, Lin Qiu, Xuezhi Cao, Xunliang Cai

机构 * Meituan Group（美团集团）； Fudan University（复旦大学）

专题命中评测与基准：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by Findings of EMNLP 2025

2509.13175 2025-09-17 cs.CV 82%

More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era

Yingtai Li, Haoran Lai, Xiaoqian Zhou, Shuai Ming, Wenxin Ma, Wei Wei, Shaohua Kevin Zhou

机构 * School of Biomedical Engineering, Division of Life Sciences ； Medicine, University of Science ； Technology of China (USTC), Hefei Anhui, 230026, China Center for Medical Imaging, Robotics, Analytic Computing \& Learning (MIRACLE), Suzhou Institute for Advance Research, USTC, Suzhou Jiangsu, 215123, China The First Affiliated Hospital of USTC, Division of Life Sciences ； Medicine, USTC, Hefei Anhui, 230001, China Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou Jiangsu, 215123, China State Key Laboratory of Precision

专题命中评测与基准：LLM(title)；large language model(abstract)；language model(abstract)

Comments MICCAI 2025

2509.12415 2025-09-17 cs.CY 82%

Prompt Commons: Collective Prompting as Governance for Urban AI

Rashid Mushkani

专题命中评测与基准：prompting(title)；large language model(abstract)；language model(abstract)

2404.01129 2025-09-17 cs.CL 81%

Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation

Bohao Yang, Kun Zhao, Dong Liu, Chen Tang, Liang Zhan, Chenghua Lin

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；SLM(abstract)

Comments EMNLP 2025 Findings

2509.13055 2025-09-17 cs.SE 80%

Automating Code Generation for Semiconductor Equipment Control from Developer Utterances with LLMs

Youngkyoung Kim, Sanghyeok Park, Misoo Kim, Gangho Yoon, Eunseok Lee, Simon S. Woo

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2509.12459 2025-09-17 cs.CL 79%

Does Language Model Understand Language?

Suvojit Acharjee, Utathya Aich, Asfak Ali

机构 * Institute of Engineering and Management（工程与管理研究所）； Jadavpur University（贾达沃大学）

专题命中评测与基准：language model(title,abstract)；分类 cs.CL

2509.12278 2025-09-17 cs.CV cs.AI 79%

PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models

Wanru Zhuang, Wenbo Li, Zhibin Lan, Xu Han, Peng Li, Jinsong Su

机构 * School of Informatics, Xiamen University, China（厦门大学信息学院）； Tsinghua, Beijing, China（清华大学）； Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）

专题命中评测与基准：language model(title,abstract)；分类 cs.AI

2507.13773 2025-09-17 cs.CV cs.CL 79%

Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions

Pu Jian, Donglei Yu, Wen Yang, Shuo Ren, Jiajun Zhang

专题命中评测与基准：language model(title,abstract)；分类 cs.CL

Comments ACL2025 Main (SAC Highlight Award)

2509.12936 2025-09-17 cs.LG cs.CL 79%

Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety

Denis Janiak, Julia Moska, Dawid Motyka, Karolina Seweryn, Paweł Walkowiak, Bartosz Żuk, Arkadiusz Janz

机构 * Wroclaw University of Science and Technology (WUST)（沃拉布大学科学与技术学院）； National Research Institute (NASK)（国家研究 institute）； Institute of Computer Science, Polish Academy of Sciences (IPI PAN)（波兰科学院计算机科学研究所）

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2509.12765 2025-09-17 cs.IR cs.AI cs.CL 79%

InfoGain-RAG: Boosting Retrieval-Augmented Generation via Document Information Gain-based Reranking and Filtering

Zihan Wang, Zihan Liang, Zhou Shao, Yufei Ma, Huangyu Dai, Ben Chen, Lingtao Mao, Chenyi Lei, Yuqing Ding, Han Li

机构 * Kuaishou Technology, Beijing, China（快手科技，北京，中国）； Peking University, Beijing, China（北京大学，北京，中国）

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP'25 Oral Presentation. Contact: benchen4395@gmail.com

2509.11967 2025-09-17 cs.LG cs.CL 79%

MillStone: How Open-Minded Are LLMs?

Harold Triedman, Vitaly Shmatikov

机构 * Cornell Tech（康奈尔科技）

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 19 pages, 7 tables, 7 figures

2509.12492 2025-09-17 cs.CV 78%

Evaluating Robustness of Vision-Language Models Under Noisy Conditions

Purushoth, Alireza

机构 * University of Nevada Reno（内华达大学拉斯维加斯分校）

专题命中评测与基准：language model(title,abstract)

2509.12405 2025-09-17 cs.CL 77%

MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering

Wen-wai Yim, Asma Ben Abacha, Zixuan Yu, Robert Doerning, Fei Xia, Meliha Yetisgen

机构 * Microsoft Health AI（微软健康人工智能）

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages, 8 tables

2505.16281 2025-09-17 cs.CL 77%

HiMATE: A Hierarchical Multi-Agent Framework for Machine Translation Evaluation

Shijie Zhang, Renhao Li, Songsheng Wang, Philipp Koehn, Min Yang, Derek F. Wong

机构 * University of Macau（澳门大学）； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）； Johns Hopkins University（约翰霍普金斯大学）

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2509.13103 2025-09-17 cs.SE 75%

Accelerating Discovery: Rapid Literature Screening with LLMs

Santiago Matalonga, Domenico Amalfitano, Jean Carlo Rossa Hauck, Martín Solari, Guilherme H. Travassos

专题命中评测与基准：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments This version of the manuscript has been submitted to Empirical Software Engieering Journal for consideration

2410.08388 2025-09-17 cs.CL cs.AI 73%

Responsible AI in NLP: GUS-Net Span-Level Bias Detection Dataset and Benchmark for Generalizations, Unfairness, and Stereotypes

Maximus Powers, Shaina Raza, Alex Chang, Rehana Riaz, Umang Mavani, Harshitha Reddy Jonala, Ansh Tiwari, Hua Wei

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2509.12750 2025-09-17 cs.CV 71%

What Makes a Good Generated Image? Investigating Human and Multimodal LLM Image Preference Alignment

Rishab Parthasarathy, Jasmine Collins, Cory Stephenson

专题命中评测与基准：LLM(title)

Comments 7 pages, 9 figures, 3 tables; appendix 16 pages, 9 figures, 6 tables

2509.13025 2025-09-17 cs.SE cs.AI 70%

GView: A Survey of Binary Forensics via Visual, Semantic, and AI-Enhanced Analysis

Raul Zaharia, Dragoş Gavriluţ, Gheorghiţă Mutu

机构 * Al. I. Cuza University \& Bitdefender, Iași, Romania

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments In Proceedings FROM 2025, arXiv:2509.11877

Journal ref EPTCS 427, 2025, pp. 134-140

2509.12961 2025-09-17 cs.CL 70%

Do LLMs Understand Wine Descriptors Across Cultures? A Benchmark for Cultural Adaptations of Wine Reviews

Chenye Zou, Xingyue Wen, Tianyi Hu, Qian Janice Wang, Daniel Hershcovich

机构 * Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系）； Department of Food and Resource Economics, University of Copenhagen（哥本哈根大学食品与资源经济学系）； Department of Food Science, University of Copenhagen（哥本哈根大学食品科学系）； Department of Computer Science, Aarhus University（阿arhus大学计算机科学系）

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

2509.12612 2025-09-17 cs.AI 70%

GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL

Daojun Chen, Xi Wang, Shenyuan Ren, Qingzhi Ma, Pengpeng Zhao, An Liu

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.AI

2509.12340 2025-09-17 cs.CL 70%

MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch

Nikolay Banar, Ehsan Lotfi, Jens Van Nooten, Cristina Arhiliuc, Marija Kliocaite, Walter Daelemans

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.CL

2509.10105 2025-09-17 cs.CV cs.CL 70%

VARCO-VISION-2.0 Technical Report

Young-rok Cha, Jeongho Ju, SunYoung Park, Jong-Hyeon Lee, Younghyun Yu, Youngjune Kim

机构 * NC AI

专题命中评测与基准：language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments 19 pages, 1 figure, 14 tables. Technical report for VARCO-VISION-2.0, a Korean-English bilingual VLM in 14B and 1.7B variants. Key features: multi-image understanding, OCR with text localization, improved Korean capabilities

2406.17962 2025-09-17 cs.CL 70%

Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework

Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Guang Yang, Chenghua Lin

机构 * The University of Manchester（曼彻斯特大学）； Tencent Timi Studio（腾讯TIMI工作室）； Durham University（杜伦大学）； University of Pittsburgh（匹兹堡大学）； University of Surrey（萨里大学）

专题命中评测与基准：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

1. 评测与基准 41 篇

EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models

Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering

Evaluating Large Language Models for Code Translation: Effects of Prompt Language and Prompt Design

Benchmarking Gender and Political Bias in Large Language Models

DaSAThco: Data-Aware SAT Heuristics Combinations Optimization via Large Language Models

PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims

IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Experts

Instance-level Randomization: Toward More Stable LLM Evaluations

More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era

Prompt Commons: Collective Prompting as Governance for Urban AI

Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation

Automating Code Generation for Semiconductor Equipment Control from Developer Utterances with LLMs

Does Language Model Understand Language?

PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models

Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions

Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety

InfoGain-RAG: Boosting Retrieval-Augmented Generation via Document Information Gain-based Reranking and Filtering

MillStone: How Open-Minded Are LLMs?

Evaluating Robustness of Vision-Language Models Under Noisy Conditions

MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering

HiMATE: A Hierarchical Multi-Agent Framework for Machine Translation Evaluation

Accelerating Discovery: Rapid Literature Screening with LLMs

Responsible AI in NLP: GUS-Net Span-Level Bias Detection Dataset and Benchmark for Generalizations, Unfairness, and Stereotypes

What Makes a Good Generated Image? Investigating Human and Multimodal LLM Image Preference Alignment

GView: A Survey of Binary Forensics via Visual, Semantic, and AI-Enhanced Analysis

Do LLMs Understand Wine Descriptors Across Cultures? A Benchmark for Cultural Adaptations of Wine Reviews

GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL

MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch

VARCO-VISION-2.0 Technical Report

Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework