语言大模型 / LLM - arXivDaily 专题

2506.04788 2025-06-06 cs.CL cs.AI cs.LG 87%

Towards LLM-Centric Multimodal Fusion: A Survey on Integration Strategies and Techniques

Jisu An, Junseok Lee, Jeoungeun Lee, Yongseok Son

机构 * Seoul National University（首尔国立大学）； University of California San Diego（加州大学圣地亚哥分校）； Chung-Ang University（Chung-Ang 大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 18 pages, 3 figures, 3 tables

2506.00482 2025-06-03 cs.LG cs.AI cs.CL 87%

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation

Eunsu Kim, Haneul Yoo, Guijin Son, Hitesh Patel, Amit Agarwal, Alice Oh

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.11705 2025-06-02 cs.CL cs.AI cs.LG cs.MA 87%

LLM Agents Making Agent Tools

Georg Wölflein, Dyke Ferber, Daniel Truhn, Ognjen Arandjelović, Jakob Nikolas Kather

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2025

2505.21689 2025-05-29 cs.CL cs.AI cs.LG 87%

LLMPR: A Novel LLM-Driven Transfer Learning based Petition Ranking Model

Avijit Gayen, Somyajit Chakraborty, Mainak Sen, Soham Paul, Angshuman Jana

机构 * India Institute of Information Technology Ghuwahati（印度信息技术学院Ghuwahati）； Techno India University（Techno India大学）； University College Cork

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 28 pages, 5 figures, journal paper, submitted to AI and Law

2501.17178 2025-05-28 cs.CL cs.AI cs.LG 87%

Tuning LLM Judge Design Decisions for 1/1000 of the Cost

David Salinas, Omar Swelam, Frank Hutter

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.02324 2025-05-26 cs.CY 87%

From Course to Skill: Evaluating LLM Performance in Curricular Analytics

Zhen Xu, Xinjin Li, Yingqi Huan, Veronica Minaya, Renzhe Yu

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2505.08414 2025-05-14 eess.IV cs.CV 87%

An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

Zhi Da Soh, Yang Bai, Kai Yu, Yang Zhou, Xiaofeng Lei, Sahil Thakur, Zann Lee, Lee Ching Linette Phang, Qingsheng Peng, Can Can Xue, Rachel Shujuan Chong, Quan V. Hoang, Lavanya Raghavan, Yih Chung Tham, Charumathi Sabanayagam, Wei-Chi Wu, Ming-Chih Ho, Jiangnan He, Preeti Gupta, Ecosse Lamoureux, Seang Mei Saw, Vinay Nangia, Songhomitra Panda-Jonas, Jie Xu, Ya Xing Wang, Xinxing Xu, Jost B. Jonas, Tien Yin Wong, Rick Siow Mong Goh, Yong Liu, Ching-Yu Cheng

专题命中评测与基准：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2502.18635 2025-05-09 cs.LG cs.AI cs.CL 87%

Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems

Matthew Barker, Andrew Bell, Evan Thomas, James Carr, Thomas Andrews, Umang Bhatt

机构 * Trustwise AI ； New York University（纽约大学）； The Alan Turing Institute（艾伦·图灵研究所）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.01711 2025-05-06 cs.CV 87%

Knowledge-Augmented Language Models Interpreting Structured Chest X-Ray Findings

Alexander Davis, Rafael Souza, Jia-Hao Lim

机构 * University of Brasilia（巴西大学）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；foundation model(abstract)

2410.03727 2025-04-28 cs.CL cs.AI cs.LG 87%

FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"

Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, Shafiq Joty

机构 * Salesforce AI Research（Salesforce AI研究院）； University of Texas at Austin（德克萨斯大学奥斯汀分校）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The conference version of this paper is published at ICLR 2025

2408.04682 2025-04-18 cs.CL cs.AI cs.LG 87%

ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities

Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming Pang

机构 * Apple（苹果公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.19178 2025-04-02 cs.IR 87%

UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering

Langming Liu, Shilei Liu, Yujin Yuan, Yizhen Zhang, Bencheng Yan, Zhiyuan Zeng, Zihao Wang, Jiaqi Liu, Di Wang, Wenbo Su, Pengjie Wang, Jian Xu, Bo Zheng

专题命中评测与基准：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 3 figures, 7 tables

2403.15426 2025-04-02 cs.LG cs.AI cs.CL 87%

CodingTeachLLM: Empowering LLM's Coding Ability via AST Prior Knowledge

Zhangquan Chen, Chunjiang Liu, Haobin Duan

机构 * Beijing OneXOne Tech Co., Ltd（北京OneXOne科技有限公司）； Tsinghua University（清华大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, 2 figures

2503.17514 2025-03-26 cs.CL cs.AI cs.CR cs.LG 87%

Language Models May Verbatim Complete Text They Were Not Explicitly Trained On

Ken Ziyu Liu, Christopher A. Choquette-Choo, Matthew Jagielski, Peter Kairouz, Sanmi Koyejo, Percy Liang, Nicolas Papernot

机构 * Google（谷歌）； Google DeepMind（谷歌DeepMind）； Stanford University（斯坦福大学）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Main text: 9 pages, 7 figures, 1 table. Appendix: 29 pages, 20 tables, 15 figures

2503.08823 2025-03-25 cs.AR cs.AI cs.CL cs.ET cs.LG 87%

ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness

Ce Guo, Tong Zhao

机构 * Imperial College London（伦敦帝国学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments to be published in International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies 2025

2503.16402 2025-03-21 cs.AI cs.CL cs.LG 87%

The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination

Yifan Sun, Han Wang, Dongbai Li, Gang Wang, Huan Zhang

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages

2503.15620 2025-03-21 cs.CL cs.AI cs.LG 87%

Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings

Austin Xu, Srijan Bansal, Yifei Ming, Semih Yavuz, Shafiq Joty

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 13 figures, 6 tables

2503.05139 2025-03-11 cs.LG cs.AI cs.CL 87%

Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs

Ling Team, Binwei Zeng, Chao Huang, Chao Zhang, Changxin Tian, Cong Chen, Dingnan Jin, Feng Yu, Feng Zhu, Feng Yuan, Fakang Wang, Gangshan Wang, Guangyao Zhai, Haitao Zhang, Huizhong Li, Jun Zhou, Jia Liu, Junpeng Fang, Junjie Ou, Jun Hu, Ji Luo, Ji Zhang, Jian Liu, Jian Sha, Jianxue Qian, Jiewei Wu, Junping Zhao, Jianguo Li, Jubao Feng, Jingchao Di, Junming Xu, Jinghua Yao, Kuan Xu, Kewei Du, Longfei Li, Lei Liang, Lu Yu, Li Tang, Lin Ju, Peng Xu, Qing Cui, Song Liu, Shicheng Li, Shun Song, Song Yan, Tengwei Cai, Tianyi Chen, Ting Guo, Ting Huang, Tao Feng, Tao Wu, Wei Wu, Xiaolu Zhang, Xueming Yang, Xin Zhao, Xiaobo Hu, Xin Lin, Yao Zhao, Yilong Wang, Yongzhen Guo, Yuanyuan Wang, Yue Yang, Yang Cao, Yuhao Fu, Yi Xiong, Yanzhe Li, Zhe Li, Zhiqiang Zhang, Ziqi Liu, Zhaoxin Huan, Zujie Wen, Zhenhang Sun, Zhuoxuan Du, Zhengyu He

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 34 pages

2503.04756 2025-03-10 cs.CY cs.AI cs.CL cs.LG 87%

Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators

Hritik Bansal, Pratyush Maini

机构 * DatologyAI（达托洛吉人工智能公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a blogpost at ICLR 2025. Originally posted at https://pratyushmaini.github.io/blog/2024/risks-private-evals/

2502.02067 2025-03-07 cs.RO cs.AI cs.CL cs.LG 87%

AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement

Shivam Singh, Karthik Swaminathan, Nabanita Dash, Ramandeep Singh, Snehasis Banerjee, Mohan Sridharan, Madhava Krishna

机构 * Robotics Research Center, IIIT Hyderabad（IIIT 海得拉巴机器人研究中心）； TCS Research, Tata Consultancy Services（塔塔咨询服务公司 TCS 研究院）； School of Informatics, University of Edinburgh（爱丁堡大学信息学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2025

2407.03157 2025-03-05 cs.CL cs.AI cs.LG cs.SE 87%

Let the Code LLM Edit Itself When You Edit the Code

Zhenyu He, Jun Zhang, Shengjie Luo, Jingjing Xu, Zhi Zhang, Di He

机构 * National Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）； School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）； ByteDance Inc.（字节跳动公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2025 Camera Ready

2408.04650 2025-03-04 cs.CL cs.AI cs.HC cs.LG 87%

Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools

Jung In Park, Mahyar Abbasian, Iman Azimi, Dawn T. Bounds, Angela Jun, Jaesu Han, Robert M. McCarron, Jessica Borelli, Parmida Safavi, Sanaz Mirbaha, Jia Li, Mona Mahmoudi, Carmen Wiedenhoeft, Amir M. Rahmani

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.21106 2025-03-03 eess.IV cs.CV 87%

A Non-contrast Head CT Foundation Model for Comprehensive Neuro-Trauma Triage

Youngjin Yoo, Bogdan Georgescu, Yanbo Zhang, Sasa Grbic, Han Liu, Gabriela D. Aldea, Thomas J. Re, Jyotipriya Das, Poikavila Ullaskrishnan, Eva Eibenberger, Andrei Chekkoury, Uttam K. Bodanapally, Savvas Nicolaou, Pina C. Sanelli, Thomas J. Schroeppel, Yvonne W. Lui, Eli Gibson

机构 * Siemens Healthineers（西门子医疗）； New York University（纽约大学）； University of Maryland Medical Center（马里兰大学医学中心）； Vancouver General Hospital（温哥华总医院）； Northwell Health（诺斯韦尔健康）； UCHealth Memorial Hospital（UCHealth纪念医院）； Siemens SRL（西门子罗马尼亚有限责任公司）； Transilvania University of Braşov（布拉索夫特兰西瓦尼亚大学）

专题命中评测与基准：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2502.21068 2025-03-03 cs.SE 87%

GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping

Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, Simone Paolo Ponzetto

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2502.20647 2025-03-03 cs.CL cs.AI cs.LG 87%

Consistency Evaluation of News Article Summaries Generated by Large (and Small) Language Models

Colleen Gilhuly, Haleh Shahzad

机构 * Royal Bank of Canada（加拿大皇家银行）； Chief Data Office（首席数据办公室）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 21 pages, 6 figures, 4 tables

2501.17187 2025-02-26 cs.CL cs.AI cs.LG 87%

Visualizing Uncertainty in Translation Tasks: An Evaluation of LLM Performance and Confidence Metrics

Jin Hyun Park, Utsawb Laminchhane, Umer Farooq, Uma Sivakumar, Arpan Kumar

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments We would like to withdraw our paper due to an error in the experimental methodology, which impacts the validity of our results. The error specifically affects the analysis presented in the Discussion, where an incorrect experimental modeling step led to misleading interpretations

2502.15740 2025-02-25 cs.SE cs.AI cs.CL cs.LG 87%

Detection of LLM-Generated Java Code Using Discretized Nested Bigrams

Timothy Paek, Chilukuri Mohan

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This preprint precedes the final peer-reviewed version, which will be published in Springer's CSCI 2024 proceedings

2502.13897 2025-02-20 cs.CL cs.AI cs.LG 87%

DataSciBench: An LLM Agent Benchmark for Data Science

Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, Yisong Yue

机构 * Tsinghua University（清华大学）； Zhipu AI（智谱AI）； University of California, Berkeley（加州大学伯克利分校）； California Institute of Technology（加州理工学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 40 pages, 7 figures, 6 tables

2501.18623 2025-02-19 cs.CV cs.GR 87%

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik

机构 * MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）； Columbia University（哥伦比亚大学）； ETH Zürich（苏黎世联邦理工学院）； Google Research（谷歌研究院）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Comments ICLR 2025 Spotlight

2411.14708 2025-02-18 cs.LG cs.AI cs.CL 87%

Understanding LLM Embeddings for Regression

Eric Tang, Bangding Yang, Xingyou Song

机构 * Stanford University（斯坦福大学）； Google（谷歌公司）； Google DeepMind（谷歌DeepMind）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in Transactions on Machine Learning Research (TMLR) 2025. Code can be found in https://github.com/google-research/optformer

1. 评测与基准 32411 篇

Towards LLM-Centric Multimodal Fusion: A Survey on Integration Strategies and Techniques

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation

LLM Agents Making Agent Tools

LLMPR: A Novel LLM-Driven Transfer Learning based Petition Ranking Model

Tuning LLM Judge Design Decisions for 1/1000 of the Cost

From Course to Skill: Evaluating LLM Performance in Curricular Analytics

An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems

Knowledge-Augmented Language Models Interpreting Structured Chest X-Ray Findings

FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"

ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities

UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering

CodingTeachLLM: Empowering LLM's Coding Ability via AST Prior Knowledge

Language Models May Verbatim Complete Text They Were Not Explicitly Trained On

ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness

The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination

Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings

Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs

Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators

AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement

Let the Code LLM Edit Itself When You Edit the Code

Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools

A Non-contrast Head CT Foundation Model for Comprehensive Neuro-Trauma Triage

GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping

Consistency Evaluation of News Article Summaries Generated by Large (and Small) Language Models

Visualizing Uncertainty in Translation Tasks: An Evaluation of LLM Performance and Confidence Metrics

Detection of LLM-Generated Java Code Using Discretized Nested Bigrams

DataSciBench: An LLM Agent Benchmark for Data Science

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Understanding LLM Embeddings for Regression