语言大模型 / LLM - arXivDaily 专题

2505.13028 2025-05-21 cs.CR cs.AI cs.CL 86%

Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Dataset

Sayon Palit, Daniel Woods

机构 * School of Informatics University of Edinburgh（信息学院爱丁堡大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2505.12808 2025-05-20 cs.CL cs.LG 86%

Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models

Yanbin Yin, Kun Zhou, Zhen Wang, Xiangdong Zhang, Yifei Shao, Shibo Hao, Yi Gu, Jieyuan Liu, Somanshu Singla, Tianyang Liu, Eric P. Xing, Zhengzhong Liu, Haojian Jin, Zhiting Hu

机构 * University of California, San Diego（加州大学圣地亚哥分校）； Mohamed bin Zayed University of AI（人工智能 Mohamed bin Zayed 大学）

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments 20 pages, ongoing work

2505.12692 2025-05-20 cs.AI cs.CL 86%

Bullying the Machine: How Personas Increase LLM Vulnerability

Ziwei Xu, Udit Sanghi, Mohan Kankanhalli

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2502.06207 2025-05-20 cs.CL cs.AI 86%

Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement

Junyu Lu, Kai Ma, Kaichun Wang, Kelaiti Xiao, Roy Ka-Wei Lee, Bo Xu, Liang Yang, Hongfei Lin

机构 * Key Laboratory of Social Computing and Cognitive Intelligence, Dalian University of Technology（大连理工大学社会计算与认知智能重点实验室）； School of Computer Science and Technology, Xinjiang Normal University（新疆师范大学计算机科学与技术学院）； Social AI Studio, Singapore University of Technology and Design（新加坡科技设计大学社会AI工作室）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, accepted at the ACL 2025

2402.14889 2025-05-19 cs.CL cs.AI 86%

COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models

Priyanshul Govil, Hemang Jain, Vamshi Krishna Bonagiri, Aman Chadha, Ponnurangam Kumaraguru, Manas Gaur, Sanorita Dey

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2503.04830 2025-05-14 cs.CL cs.AI 86%

Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agents

Jingying Zeng, Hui Liu, Zhenwei Dai, Xianfeng Tang, Chen Luo, Samarth Varshney, Zhen Li, Qi He

机构 * Amazon（亚马逊）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2505.05612 2025-05-12 cs.AI cs.LG q-bio.QM 86%

scDrugMap: Benchmarking Large Foundation Models for Drug Response Prediction

Qing Wang, Yining Pan, Minghao Zhou, Zijia Tang, Yanfei Wang, Guangyu Wang, Qianqian Song

专题命中评测与基准：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 14 pages, 7 figures

2401.15127 2025-05-07 cs.CR cs.CL cs.LG 86%

Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness

Samaneh Shafee, Alysson Bessani, Pedro M. Ferreira

机构 * LASIGE, Faculdade de Ciências, Universidade de Lisboa（LASIGE，科学学院，里斯本大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Journal ref Expert Systems with Applications, Volume 261, 1 February 2025, 125509

2410.14567 2025-05-06 cs.CL cs.AI cs.IR 86%

ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions

Zhiyuan Peng, Jinming Nian, Alexandre Evfimievski, Yi Fang

机构 * Santa Clara University（圣克拉拉大学）； Adobe Inc.（Adobe公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by SIGIR'25

2504.21851 2025-05-01 cs.CL cs.AI 86%

TRUST: An LLM-Based Dialogue System for Trauma Understanding and Structured Assessments

Sichang Tu, Abigail Powers, Stephen Doogan, Jinho D. Choi

机构 * DooGood Foundation（DooGood基金会）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 5 figures, 4 tables

2504.21605 2025-05-01 cs.CL cs.AI cs.IR 86%

RDF-Based Structured Quality Assessment Representation of Multilingual LLM Evaluations

Jonas Gwozdz, Andreas Both

机构 * Leipzig University of Applied Sciences（莱比锡应用科学大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.21032 2025-05-01 cs.CY cs.AI cs.LG 86%

Selecting the Right LLM for eGov Explanations

Lior Limonad, Fabiana Fournier, Hadar Mulian, George Manias, Spiros Borotis, Danai Kyrkou

机构 * IBM Research, Israel（IBM以色列研究院）； Department of Digital Systems, University of Piraeus（皮雷奥斯大学数字系统系）； Maggioli S.p.A.（Maggioli公司）； ViLabs

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 7 figures. ICEDEG 2025, Bern, Switzerland, June 2025

2504.17550 2025-04-25 cs.CL cs.AI 86%

HalluLens: LLM Hallucination Benchmark

Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung

机构 * FAIR at Meta（Meta的FAIR部门）； GenAI at Meta（Meta的GenAI部门）； HKUST（香港科技大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 42 pages

2504.15205 2025-04-22 cs.CL cs.AI cs.IR 86%

Support Evaluation for the TREC 2024 RAG Track: Comparing Human versus LLM Judges

Nandan Thakur, Ronak Pradeep, Shivani Upadhyay, Daniel Campos, Nick Craswell, Jimmy Lin

机构 * University of Waterloo（滑铁卢大学）； Snowflake（Snowflake公司）； Microsoft（微软公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at SIGIR 2025 (short)

2504.14039 2025-04-22 cs.CL cs.AI 86%

MEQA: A Meta-Evaluation Framework for Question & Answer LLM Benchmarks

Jaime Raldua Veuthey, Zainab Ali Majid, Suhas Hariharan, Jacob Haimes

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.11363 2025-04-21 cs.AI cs.CE cs.LG q-bio.BM 86%

ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, Wei Wang

机构 * University of California, Los Angeles（加利福尼亚大学洛杉矶分校）； Georgia Institute of Technology（佐治亚理工学院）； Meta AI

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Spotlight, Machine Learning for Genomics Explorations @ ICLR 2025

2504.12350 2025-04-18 cs.CL cs.AI 86%

A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reports

Jing Wang, Jeremy C Weiss

专题命中评测与基准：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Journal ref 2025 AMIA Informatics Summit Proceedings, March 10-13, Pittsburgh, PA

2503.10673 2025-04-18 cs.CL cs.AI 86%

ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition

Hisham A. Alyahya, Haidar Khan, Yazeed Alnumay, M Saiful Bari, Bülent Yener

机构 * Saudi Data & AI Authority (SDAIA)（沙特数据与人工智能局）； Meta ； Cohere（Cohere公司）； Rensselaer Polytechnic Institute（伦斯勒理工学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2402.16063 2025-04-18 cs.CL cs.AI 86%

Citation-Enhanced Generation for LLM-based Chatbots

Weitao Li, Junkai Li, Weizhi Ma, Yang Liu

机构 * Tsinghua University（清华大学）； Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）； Jiangsu Collaborative Innovation Center for Language Competence（江苏省语言能力协同创新中心）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref Proc. 62nd ACL Vol. 1 Long Papers, Bangkok Thailand, pp. 1451-1466, 2024

2503.16514 2025-04-17 cs.AR cs.AI cs.LG cs.PL 86%

VeriMind: Agentic LLM for Automated Verilog Generation with a Novel Evaluation Metric

Bardia Nadimi, Ghali Omar Boutaib, Hao Zheng

机构 * University of South Florida（南佛罗里达大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2504.10179 2025-04-15 cs.AI cs.CL cs.ET 86%

The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance

Anwesha Mohanty, Venkatesh Balavadhani Parthasarathy, Arsalan Shahid

机构 * University College Dublin（都柏林大学）； CeADAR: Ireland’s Centre for AI（爱尔兰人工智能与数据中心（CeADAR））

专题命中评测与基准：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.09522 2025-04-15 cs.CL cs.AI 86%

How new data permeates LLM knowledge and how to dilute it

Chen Sun, Renat Aksitov, Andrey Zhmoginov, Nolan Andrew Miller, Max Vladymyrov, Ulrich Rueckert, Been Kim, Mark Sandler

机构 * Google DeepMind（谷歌DeepMind）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.02479 2025-04-15 cs.SE cs.AI cs.CL 86%

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen

机构 * School of Electrical and Computer Engineering, The University of Sydney（悉尼大学电气与计算机工程学院）； University at Buffalo, SUNY（纽约州立大学布法罗分校）； University of Chicago（芝加哥大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.07278 2025-04-11 cs.LG cs.AI 86%

A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment, and LLM Automation

Fatemeh Amrollahi, Nicholas Marshall, Fateme Nateghi Haredasht, Kameron C Black, Aydin Zahedivash, Manoj V Maddali, Stephen P. Ma, Amy Chang, MD Phar Stanley C Deresinski, Mary Kane Goldstein, Steven M. Asch, Niaz Banaei, Jonathan H Chen

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 2 figures, 2 tables, conference

2504.06600 2025-04-10 cs.CL cs.AI cs.SE 86%

Automated Business Process Analysis: An LLM-Based Approach to Value Assessment

William De Michele, Abel Armas Cervantes, Lea Frermann

机构 * The University of Melbourne（墨尔本大学）

专题命中评测与基准：LLM(title)；large language model(abstract)；language model(abstract)；prompting(abstract)

2406.15341 2025-04-09 cs.LG cs.AI q-bio.GN 86%

GenoTEX: An LLM Agent Benchmark for Automated Gene Expression Data Analysis

Haoyang Liu, Shuyu Chen, Ye Zhang, Haohan Wang

机构 * Novartis（诺华公司）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 31 pages, 4 figures

2504.04314 2025-04-08 cs.CL cs.AI math.ST stat.TH 86%

Balancing Complexity and Informativeness in LLM-Based Clustering: Finding the Goldilocks Zone

Justin Miller, Tristram Alexander

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 4 figures, 2 tables

2412.18702 2025-04-08 cs.CL cs.AI cs.DB 86%

CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Era

Yanlin Feng, Simone Papicchio, Sajjadur Rahman

机构 * Megagon Labs（美伽仑实验室）； Politecnico di Torino（都灵理工大学）； EURECOM（欧洲通信学院）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.02891 2025-04-07 cs.CL cs.AI 86%

Automated Survey Collection with LLM-based Conversational Agents

Kurmanbek Kaiyrbekov, Nicholas J Dobbins, Sean D Mooney

机构 * National Human Genome Research Institute, National Institutes of Health（美国国立卫生研究院国家人类基因组研究所）； Johns Hopkins University（约翰斯·霍普金斯大学）

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.02867 2025-04-07 cs.CL cs.AI 86%

Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications

Hongliu Cao, Ilias Driouich, Robin Singh, Eoin Thomas

机构 * Amadeus SAS

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Presented at SophiaSummit2024

1. 评测与基准 32413 篇

Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Dataset

Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models

Bullying the Machine: How Personas Increase LLM Vulnerability

Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement

COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models

Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agents

scDrugMap: Benchmarking Large Foundation Models for Drug Response Prediction

Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness

ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions

TRUST: An LLM-Based Dialogue System for Trauma Understanding and Structured Assessments

RDF-Based Structured Quality Assessment Representation of Multilingual LLM Evaluations

Selecting the Right LLM for eGov Explanations

HalluLens: LLM Hallucination Benchmark

Support Evaluation for the TREC 2024 RAG Track: Comparing Human versus LLM Judges

MEQA: A Meta-Evaluation Framework for Question & Answer LLM Benchmarks

ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reports

ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition

Citation-Enhanced Generation for LLM-based Chatbots

VeriMind: Agentic LLM for Automated Verilog Generation with a Novel Evaluation Metric

The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance

How new data permeates LLM knowledge and how to dilute it

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment, and LLM Automation

Automated Business Process Analysis: An LLM-Based Approach to Value Assessment

GenoTEX: An LLM Agent Benchmark for Automated Gene Expression Data Analysis

Balancing Complexity and Informativeness in LLM-Based Clustering: Finding the Goldilocks Zone

CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Era

Automated Survey Collection with LLM-based Conversational Agents

Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications