语言大模型 / LLM - arXivDaily 专题

2510.03029 2025-10-06 cs.SE cs.AI 85%

Investigating The Smells of LLM Generated Code

Debalina Ghosh Paul, Hong Zhu, Ian Bayley

机构 * School of Engineering, Computing and Mathematics, Oxford Brookes University（工程、计算与数学学院，奥克斯福德布鲁克斯大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.01552 2025-10-03 cs.CR cs.AI 85%

POLAR: Automating Cyber Threat Prioritization through LLM-Powered Assessment

Luoxi Tang, Yuqiao Meng, Ankita Patra, Weicheng Ma, Muchao Ye, Zhaohan Xi

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 25 pages

2510.01255 2025-10-03 cs.CL cs.CY cs.HC 85%

Longitudinal Monitoring of LLM Content Moderation of Social Issues

Yunlang Dai, Emma Lurie, Danaé Metaxa, Sorelle A. Friedler

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2510.00844 2025-10-02 cs.AI 85%

Learning Compact Representations of LLM Abilities via Item Response Theory

Jianhao Chen, Chenxu Wang, Gengrui Zhang, Peng Ye, Lei Bai, Wei Hu, Yuzhong Qu, Shuyue Hu

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）； Fudan University（复旦大学）； Department of Psychology, University of Southern California（美国南加州大学心理学系）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.00797 2025-10-02 cs.CV cs.AI 85%

Solar PV Installation Potential Assessment on Building Facades Based on Vision and Language Foundation Models

Ruyu Liu, Dongxu Zhuang, Jianhua Zhang, Arega Getaneh Abate, Per Sieverts Nielsen, Ben Wang, Xiufeng Liu

机构 * Technical University of Denmark（技术大学）

专题命中评测与基准：foundation model(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

2510.00324 2025-10-02 cs.SE cs.IR cs.LG 85%

Which Programming Language and Model Work Best With LLM-as-a-Judge For Code Retrieval?

Lucas Roberts, Denisa Roberts

机构 * Independent Researcher（独立研究者）； New York University（纽约大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Accepted as a full paper at SIGIR-AP 2025

2509.19136 2025-10-02 cs.SE cs.AI 85%

On the Soundness and Consistency of LLM Agents for Executing Test Cases Written in Natural Language

Sébastien Salva, Redha Taguelmimt

机构 * LIMOS - UMR CNRS 6158, Clermont Auvergne University, UCA, Aubière, France（克莱蒙特奥弗涅大学）； Department of Computing, Main University, MySecondTown, MyCountry（计算机系，主大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2509.18980 2025-10-02 cs.AI cs.HC cs.IR 85%

From latent factors to language: a user study on LLM-generated explanations for an inherently interpretable matrix-based recommender system

Maxime Manderlier, Fabian Lecron, Olivier Vu Thanh, Nicolas Gillis

机构 * Department of Technological Innovation Management, Faculty of Engineering, University of Mons (UMONS), Mons, Belgium（蒙斯大学工程学院技术创新管理系）； Department of Mathematics and Operational Research, Faculty of Engineering, University of Mons (UMONS), Mons, Belgium（蒙斯大学工程学院数学与运筹学系）； Department of Mathematics（数学系）； Operational Research, Faculty of Engineering, University of Mons (UMONS), Mons, Belgium（运筹学系）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Journal ref In Proceedings of the 12th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS 2025) co-located with 19th ACM Conference on Recommender Systems (RecSys 2025)

2509.25248 2025-10-01 cs.SE cs.AI cs.PL 85%

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

Zehua Zhang, Ati Priya Bajaj, Divij Handa, Siyu Liu, Arvind S Raj, Hongkai Chen, Hulin Wang, Yibo Liu, Zion Leonahenahe Basque, Souradip Nath, Vishal Juneja, Nikhil Chapre, Yan Shoshitaishvili, Adam Doupé, Chitta Baral, Ruoyu Wang

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2509.25154 2025-09-30 cs.AI 85%

Who's Your Judge? On the Detectability of LLM-Generated Judgments

Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang, Pingchuan Ma, Abdullah Alnaibari, Kai Shu, Huan Liu

机构 * Arizona State University（亚利桑那州立大学）； Emory University（埃默里大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Under review

2507.04952 2025-09-30 cs.CL cs.SE 85%

ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation

Chenchen Zhang, Yuhang Li, Can Xu, Jiaheng Liu, Ao Liu, Changzhi Zhou, Ken Deng, Dengpeng Wu, Guanhua Huang, Kejiao Li, Qi Yi, Ruibin Xiong, Shihui Hu, Yue Zhang, Yuhao Jiang, Zenan Xu, Yuanxing Zhang, Wiggin Zhou, Chayse Zhou, Fengzong Lian

机构 * Tencent Hunyuan Team（腾讯文脉团队）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2509.23510 2025-09-30 cs.AI 85%

Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores

Ashwin Ramaswamy, Nestor Demeure, Ermal Rrapaj

机构 * Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2509.22391 2025-09-29 cs.AI 85%

Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents

Jiaqi Shao, Yuxiang Lin, Munish Prasad Lohani, Yufeng Miao, Bing Luo

机构 * Duke Kunshan University（杜克昆山大学）； Microsoft AI（微软人工智能）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2509.22243 2025-09-29 cs.CL 85%

FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction

Yuan Ge, Saihan Chen, Jingqi Xiao, Xiaoqian Liu, Tong Xiao, Yan Xiang, Zhengtao Yu, Jingbo Zhu

机构 * School of Computer Science and Engineering, Northeastern University, Shenyang, China（计算机科学与工程学院，东北大学，沈阳，中国）； NiuTrans Research（NiuTrans研究）； Kunming University of Science and Technology（昆明理工大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2509.21978 2025-09-29 cs.CL 85%

MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation

Xinping Lei, Tong Zhou, Yubo Chen, Kang Liu, Jun Zhao

机构 * The Key Laboratory of Cognition and Decision Intelligence for Complex Systems（认知与决策智能复杂系统重点实验室）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）； Hunan Provincial Key Laboratory of Philosophy and Social Sciences of Artificial Intelligence and Precision International, Hunan Normal University（湖南省人工智能与精准哲学社会科学省级重点实验室，湖南师范大学）； Beijing University of Posts and Telecommunications（北京邮电大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP2025 Findings

2503.03064 2025-09-29 cs.CL 85%

Improving LLM-as-a-Judge Inference with the Judgment Distribution

Victor Wang, Michael J. Q. Zhang, Eunsol Choi

机构 * Department of Computer Science（计算机科学系）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）； New York University（纽约大学）

专题命中评测与基准：LLM(title,abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

2410.03981 2025-09-29 cs.SE cs.LG 85%

A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages

Sathvik Joel, Jie JW Wu, Fatemeh H. Fard

机构 * Indian Institute of Technology Madras, Chennai（印度理工学院马德拉斯分校，钦奈）； University of British Columbia, Kelowna（不列颠哥伦比亚大学，克洛维纳）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 64 pages, 3 figures, 15 tables. Accepted in ACM Transactions on Software Engineering and Methodology (TOSEM)

2509.19347 2025-09-25 cs.CL 85%

Characterizing Knowledge Graph Tasks in LLM Benchmarks Using Cognitive Complexity Frameworks

Sara Todorovikj, Lars-Peter Meyer, Michael Martin

机构 * Chemnitz University of Technology, Germany（德累斯顿技术大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments peer reviewed publication at SEMANTiCS 2025 Poster Track

2509.19329 2025-09-25 cs.CL stat.ME 85%

How Model Size, Temperature, and Prompt Style Affect LLM-Human Assessment Score Alignment

Julie Jung, Max Lu, Sina Chole Benker, Dogus Darici

机构 * Harvard Graduate School of Education（哈佛教育研究生院）； Munster University（穆恩斯特大学）； Institute of Anatomy and Neurobiology, University of Münster（解剖与神经生物学研究所，穆恩斯特大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages, 4 figures, accepted at NCME AIME 2025

2509.19117 2025-09-24 cs.CR cs.LG cs.SE 85%

LLM-based Vulnerability Discovery through the Lens of Code Metrics

Felix Weissberg, Lukas Pirch, Erik Imgrund, Jonas Möller, Thorsten Eisenhofer, Konrad Rieck

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2509.18575 2025-09-24 cs.IR cs.AI 85%

The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking

Yaoyao Qian, Yifan Zeng, Yuchao Jiang, Chelsi Jain, Huazheng Wang

机构 * Northeastern University（东北大学）； Oregon State University（俄勒冈州立大学）； University of Macau（澳门大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted by EMNLP 2025

2509.16516 2025-09-24 cs.LG 85%

LLM-Guided Co-Training for Text Classification

Md Mezbaur Rahman, Cornelia Caragea

机构 * Computer Science University of Illinois Chicago（计算机科学伊利诺伊大学芝加哥分校）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2509.15260 2025-09-24 cs.CL 85%

Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages

Yujia Hu, Ming Shan Hee, Preslav Nakov, Roy Ka-Wei Lee

机构 * Singapore University of Technology and Design（新加坡科技设计大学）； Mohamed bin Zayed University of Artificial Intelligence（马尔代夫 bin Zayed 人工智能大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages, EMNLP 2025

2509.17436 2025-09-23 cs.CL 85%

MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses

Tong Chen, Zimu Wang, Yiyi Miao, Haoran Luo, Yuanfei Sun, Wei Wang, Zhengyong Jiang, Procheta Sen, Jionglong Su

机构 * School of AI and Advanced Computing, Xi’an Jiaotong-Liverpool University, China（人工智能与高级计算学院，西安交通大学利物浦大学，中国）； School of Advanced Technology, Xi’an Jiaotong-Liverpool University, China（先进技术学院，西安交通大学利物浦大学，中国）； Department of Computer Science, University of Liverpool, United Kingdom（计算机科学系，利物浦大学，英国）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2025. Camera-ready version

2509.16861 2025-09-23 cs.CR cs.AI cs.SE 85%

AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software

Rui Yang, Michael Fu, Chakkrit Tantithamthavorn, Chetan Arora, Gunel Gulmammadova, Joey Chua

机构 * Monash University（墨尔本大学）； The University of Melbourne（墨尔本大学）； Transurban

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to the ASE 2025 International Conference on Automated Software Engineering, Industry Showcase Track

2407.12797 2025-09-23 cs.PF cs.LG 85%

CEBench: A Benchmarking Toolkit for the Cost-Effectiveness of LLM Pipelines

Wenbo Sun, Jiaqi Wang, Qiming Guo, Ziyu Li, Wenlu Wang, Rihan Hai

机构 * Delft University of Technology（代尔夫特理工大学）； Texas A&M University - Corpus Christi（德克萨斯A&M大学-科罗萨多）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2504.09407 2025-09-22 cs.CL cs.HC 85%

UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents

Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang

机构 * Northeastern University（东北大学）； Amazon（亚马逊）； University of Notre Dame（诺特尔大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2505.19477 2025-09-19 cs.AI 85%

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge

Chiyu Ma, Enpei Zhang, Yilun Zhao, Wenjun Liu, Yaning Jia, Peijun Qing, Lin Shi, Arman Cohan, Yujun Yan, Soroush Vosoughi

机构 * Dartmouth College（达特茅斯学院）； Yale University（耶鲁大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2505.19176 2025-09-19 cs.CL 85%

Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge

Zhuo Liu, Moxin Li, Xun Deng, Qifan Wang, Fuli Feng

机构 * University of Science and Technology of China（中国科学技术大学）； National University of Singapore（新加坡国立大学）； Meta AI

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Under review

2505.06469 2025-09-19 cs.AI cs.HC 85%

KCluster: An LLM-based Clustering Approach to Knowledge Component Discovery

Yumou Wei, Paulo Carvalho, John Stamper

机构 * Human-Computer Interaction Institute（人机交互研究所）； Carnegie Mellon University（卡内基梅隆大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to the Educational Data Mining (EDM) 2025 conference

1. 评测与基准 32459 篇

Investigating The Smells of LLM Generated Code

POLAR: Automating Cyber Threat Prioritization through LLM-Powered Assessment

Longitudinal Monitoring of LLM Content Moderation of Social Issues

Learning Compact Representations of LLM Abilities via Item Response Theory

Solar PV Installation Potential Assessment on Building Facades Based on Vision and Language Foundation Models

Which Programming Language and Model Work Best With LLM-as-a-Judge For Code Retrieval?

On the Soundness and Consistency of LLM Agents for Executing Test Cases Written in Natural Language

From latent factors to language: a user study on LLM-generated explanations for an inherently interpretable matrix-based recommender system

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

Who's Your Judge? On the Detectability of LLM-Generated Judgments

ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation

Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores

Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents

FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction

MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation

Improving LLM-as-a-Judge Inference with the Judgment Distribution

A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages

Characterizing Knowledge Graph Tasks in LLM Benchmarks Using Cognitive Complexity Frameworks

How Model Size, Temperature, and Prompt Style Affect LLM-Human Assessment Score Alignment

LLM-based Vulnerability Discovery through the Lens of Code Metrics

The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking

LLM-Guided Co-Training for Text Classification

Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages

MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses

AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software

CEBench: A Benchmarking Toolkit for the Cost-Effectiveness of LLM Pipelines

UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge

Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge

KCluster: An LLM-based Clustering Approach to Knowledge Component Discovery