语言大模型 / LLM - arXivDaily 专题

2506.11452 2025-12-03 cs.IR 89%

Leveraging Reference Documents for Zero-Shot Ranking via Large Language Models

通过大型语言模型利用参考文档实现零样本排序

Jieran Li, Xiuyuan Hu, Yang Zhao, Shengyao Zhuang, Hao Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结 RefRank 通过利用固定参考文档实现零样本排序，以线性时间复杂度提升排序准确性并降低计算成本。

2511.22047 2025-12-01 cs.CR 89%

Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks

评估大型语言模型安全防护措施对对抗攻击的鲁棒性

Richard J. Young

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结本研究评估了大型语言模型安全防护措施对对抗攻击的鲁棒性，发现模型在未见过的提示上表现显著下降，且存在新的失败模式，强调泛化能力的重要性。

Comments 21 pages, 9 figures, 6 tables

2511.22016 2025-12-01 cs.CL cs.AI cs.LG 89%

AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models

AfriStereo：一种基于文化背景的数据集，用于评估大型语言模型中的刻板印象偏见

Yann Le Beux, Oluchi Audu, Oche D. Ankeli, Dhananjay Balakrishnan, Melissah Weya, Marie D. Ralaiarinosy, Ignatius Ezeani

专题命中评测与基准：language model(title,abstract)；large language model(title)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 AfriStereo通过构建基于非洲文化的数据集，评估大型语言模型中的刻板印象偏见，揭示了模型在性别、年龄、职业等维度上的系统性偏见。

2511.09606 2025-11-27 cs.CR 89%

How Can We Effectively Use LLMs for Phishing Detection?: Evaluating the Effectiveness of Large Language Model-based Phishing Detection Models

如何有效利用大语言模型进行钓鱼检测？：评估基于大语言模型的钓鱼检测模型的有效性

Fujiao Ji, Doowon Kim

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结本文评估了基于大语言模型的钓鱼检测有效性，发现商业模型在检测中表现更优，截图输入在品牌识别中效果最佳，但需注意温度设置和多模态输入的影响。

2409.08087 2025-11-26 cs.CR 89%

Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks

保障大语言模型：应对偏见、虚假信息和提示攻击

Benji Peng, Keyu Chen, Ming Li, Pohsun Feng, Ziqian Bi, Junyu Liu, Xinyuan Song, Qian Niu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结本文探讨了大语言模型在偏见、虚假信息和提示攻击方面的安全问题，分析了偏见缓解策略、内容检测机制及防御措施，强调了对LLM安全领域进一步研究的重要性。

Comments 17 pages, 1 figure

2511.11410 2025-11-17 cs.CV 89%

Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models

Jiaxi Huang, Dongxu Wu, Hanwei Zhu, Lingyu Zhu, Jun Xing, Xu Wang, Baoliang Chen

机构 * South China Normal University（华南师范大学）； Nanyang Technological University（南洋理工大学）； City University of Hong Kong（香港城市大学）； Shenzhen Academy of Inspection and Quarantine（深圳检验检疫局）； Shenzhen University（深圳大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

2511.10418 2025-11-14 cs.DB 89%

CityVerse: A Unified Data Platform for Multi-Task Urban Computing with Large Language Models

Yaqiao Zhu, Hongkai Wen, Mark Birkin, Man Luo

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2508.11034 2025-11-14 cs.SE 89%

The Impact of Large Language Models (LLMs) on Code Review Process

Antonio Collante, Samuel Abedu, SayedHassan Khatoonabadi, Ahmad Abdellatif, Ebube Alor, Emad Shihab

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2507.03739 2025-11-11 cs.CV 89%

ChestGPT: Integrating Large Language Models and Vision Transformers for Disease Detection and Localization in Chest X-Rays

Shehroz S. Khan, Petar Przulj, Ahmed Ashraf, Ali Abedi

机构 * College of Engineering and Technology, American University of the Middle East（工程与技术学院，中东大学）； Faculty of Applied Science and Engineering, University of Toronto（应用科学与工程学院，多伦多大学）； Department of Electrical and Computer Engineering, University of Manitoba（电气与计算机工程系，曼尼托巴大学）； KITE Research Institute, Toronto Rehabilitation Institute, University Health Network（KITE研究机构，多伦多康复研究所，大学健康网络）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 8 pages, 5 figures, 4 tables

2505.23262 2025-11-10 cs.CY 89%

Applying Large Language Models to Travel Satisfaction Analysis

Pengfei Xu, Donggen Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2511.04267 2025-11-07 cs.SE 89%

A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios

Jiahui Wu, Chengjie Lu, Aitor Arrieta, Shaukat Ali

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2505.18658 2025-11-07 cs.CL cs.AI cs.LG 89%

Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics

Pankaj Kumar, Subhankar Mishra

机构 * School of Computer Sciences（计算机科学学院）； National Institute of Science Education and Research（国家科学教育与研究 institute）； An OCC of Homi Bhabha National Institute, India（霍米·巴布国家研究所办公室，印度）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at TMLR

2509.12593 2025-11-06 cs.PL 89%

Converting IEC 61131-3 LD into SFC Using Large Language Model: Dataset and Testing

Yimin Zhang, Mario de Sousa

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2505.08903 2025-11-05 cs.SE 89%

Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks

Xing Hu, Feifei Niu, Junkai Chen, Xin Zhou, Junwei Zhang, Junda He, Xin Xia, David Lo

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2511.00624 2025-11-04 cs.SE 89%

Can Large Language Models Detect Real-World Android Software Compliance Violations?

Haoyi Zhang, Huaijin Ran, Xunzhu Tang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2510.08604 2025-10-31 cs.CL cs.AI cs.LG 89%

LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback

Raffaele Mura, Giorgio Piras, Kamilė Lukošiūtė, Maura Pintor, Amin Karbasi, Battista Biggio

机构 * University of Cagliari（卡利亚里大学）； Centre for AI Governance（人工智能治理中心）； Foundation AI – Cisco Systems Inc.（AI基金会——思科系统公司）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2505.20945 2025-10-31 cs.CR 89%

IRCopilot: Automated Incident Response with Large Language Models

Xihuan Lin, Jie Zhang, Gelei Deng, Tianzhe Liu, Tianwei Zhang, Qing Guo, Riqing Chen

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2408.15066 2025-10-31 cs.HC cs.CY 89%

Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models

Ned Cooper, Alexandra Zafiroglu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments Version accepted for publication

Journal ref ACM Journal on Responsible Computing, Volume 2, Issue 4, Article 16 (2025)

2412.15584 2025-10-30 cs.HC 89%

To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models

Jessica Y. Bo, Sophia Wan, Ashton Anderson

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Journal ref Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

2510.25148 2025-10-30 cs.SE 89%

Automated Program Repair Based on REST API Specifications Using Large Language Models

Katsuki Yamagishi, Norihiro Yoshida, Erina Makihara, Katsuro Inoue

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2410.15817 2025-10-30 cs.CE 89%

LaMP-Val: Large Language Models Empower Personalized Valuation in Auction

Jie Sun, Tianyu Zhang, Houcheng Jiang, Kexin Huang, Xiang Shu, Zhibo Zhu, Lintao Ma, Xingyu Lu, Jun Zhou, Junkang Wu, Chi Luo, An Zhang, Junkang Wu, Jiancan Wu, Xiang Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 17 pages, 5 figures, 8tables

2510.18936 2025-10-28 cs.IR cs.SE 89%

SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Mining

Hamed Jelodar, Mohammad Meymani, Samita Bai, Roozbeh Razavi-Far, Ali A. Ghorbani

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2508.11871 2025-10-27 cs.NE 89%

LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization

Zhen-Song Chen, Hong-Wei Ding, Xian-Jia Wang, Witold Pedrycz

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments This work has been submitted to the IEEE for possible publication

2502.19573 2025-10-27 cs.CL cs.AI cs.LG 89%

Do Large Language Models Know How Much They Know?

Gabriele Prato, Jerry Huang, Prasanna Parthasarathi, Shagun Sodhani, Sarath Chandar

机构 * Chandar Research Lab（昌达尔研究实验室）； Mila – Quebec AI Institute（魁北克人工智能研究所）； Université de Montréal（蒙特利尔大学）； Meta FAIR ； Polytechnique Montréal（蒙特利尔理工学院）； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a long paper at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Official version of paper within conference proceedings is available at https://aclanthology.org/2024.emnlp-main.348/

2306.13394 2025-10-27 cs.CV 89%

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, Ran He

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）； School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）； Tencent Youtu Lab（腾讯优图实验室）； Xiamen University（厦门大学）； CASIA（中国科学院自动化研究所）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments NeurIPS DB 2025 Spotlight, Project Page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation

2510.18456 2025-10-22 cs.SE 89%

Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research

Cristina Martinez Montes, Robert Feldt, Cristina Miguel Martos, Sofia Ouhbi, Shweta Premanandan, Daniel Graziotin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2405.10255 2025-10-22 cs.CV cs.RO 89%

When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models

Xianzheng Ma, Brandon Smart, Yash Bhalgat, Shuai Chen, Xinghui Li, Jian Ding, Jindong Gu, Dave Zhenyu Chen, Songyou Peng, Jia-Wang Bian, Philip H Torr, Marc Pollefeys, Matthias Nießner, Ian D Reid, Angel X. Chang, Iro Laina, Victor Adrian Prisacariu

机构 * University of Oxford（牛津大学）； King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学）； Technical University of Munich（慕尼黑技术大学）； Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）； Simon Fraser University（西蒙·弗雷泽大学）； ETH Zurich（苏黎世联邦理工学院）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 2nd version update to Jun.2025

2507.05289 2025-10-21 cs.SE 89%

Measuring how changes in code readability attributes affect code quality evaluation by Large Language Models

Igor Regis da Silva Simoes, Elaine Venson

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Journal ref 2025: Proceedings of the XXXIX Brazilian Symposium on Software Engineering

2510.15260 2025-10-20 cs.LG cs.AI cs.CL 89%

DRO-InstructZero: Distributionally Robust Prompt Optimization for Large Language Models

Yangyang Li

机构 * Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）； Massachusetts Institute of Technology（麻省理工学院）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint. Under review at ICLR 2026. 11 pages, 2 figures

2510.13812 2025-10-17 cs.HC 89%

MindBenchAI: An Actionable Platform to Evaluate the Profile and Performance of Large Language Models in a Mental Healthcare Context

Bridget Dwyer, Matthew Flathers, Akane Sano, Allison Dempsey, Andrea Cipriani, Asim H. Gazi, Carla Gorban, Carolyn I. Rodriguez, Charles Stromeyer, Darlene King, Eden Rozenblit, Gillian Strudwick, Jake Linardon, Jiaee Cheong, Joseph Firth, Julian Herpertz, Julian Schwarz, Margaret Emerson, Martin P. Paulus, Michelle Patriquin, Yining Hua, Soumya Choudhary, Steven Siddals, Laura Ospina Pinillos, Jason Bantjes, Steven Scheuller, Xuhai Xu, Ken Duckworth, Daniel H. Gillison, Michael Wood, John Torous

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

1. 评测与基准 32096 篇

Leveraging Reference Documents for Zero-Shot Ranking via Large Language Models

Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks

AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models

How Can We Effectively Use LLMs for Phishing Detection?: Evaluating the Effectiveness of Large Language Model-based Phishing Detection Models

Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks

Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models

CityVerse: A Unified Data Platform for Multi-Task Urban Computing with Large Language Models

The Impact of Large Language Models (LLMs) on Code Review Process

ChestGPT: Integrating Large Language Models and Vision Transformers for Disease Detection and Localization in Chest X-Rays

Applying Large Language Models to Travel Satisfaction Analysis

A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios

Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics

Converting IEC 61131-3 LD into SFC Using Large Language Model: Dataset and Testing

Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks

Can Large Language Models Detect Real-World Android Software Compliance Violations?

LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback

IRCopilot: Automated Incident Response with Large Language Models

Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models

To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models

Automated Program Repair Based on REST API Specifications Using Large Language Models

LaMP-Val: Large Language Models Empower Personalized Valuation in Auction

SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Mining

LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization

Do Large Language Models Know How Much They Know?

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research

When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models

Measuring how changes in code readability attributes affect code quality evaluation by Large Language Models

DRO-InstructZero: Distributionally Robust Prompt Optimization for Large Language Models

MindBenchAI: An Actionable Platform to Evaluate the Profile and Performance of Large Language Models in a Mental Healthcare Context