语言大模型 / LLM

evalSmarT: An LLM-Based Framework for Evaluating Smart Contract Generated Comments

Fatou Ndiaye Mbodji

机构 * SnT – University of Luxembourg（卢森堡大学SnT学院） ； Université Cheikh Anta Diop（谢赫·安塔·迪奥普大学） ； Huawei（华为） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 4 pages, 4 tables

MCP4EDA: LLM-Powered Model Context Protocol RTL-to-GDSII Automation with Backend Aware Synthesis Optimization

Yiting Wang, Wanghao Ye, Yexiao He, Yiran Chen, Gang Qu, Ang Li

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract,comments)；language model(abstract,comments)

Comments 7 pages, 5 figures Keywords: Model Context Protocol, Electronic Design Automation, Large Language Models, Synthesis Optimization

MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them

Weichen Zhang, Yiyou Sun, Pohao Huang, Jiayue Pu, Heyue Lin, Dawn Song

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Code and data: https://github.com/sunblaze-ucb/mirage-bench.git

MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models

Hafsteinn Einarsson

机构 * University of Iceland（爱沙尼亚大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.AI

KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Models

Seorin Kim, Dongyoung Lee, Jaejin Lee

机构 * Dept. of Data Science, Seoul National University（数据科学系，首尔国立大学） ； Dept. of Computer Science and Engineering, Seoul National University（计算机科学与工程系，首尔国立大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；prompting(abstract)；分类 cs.CL

Checklist Engineering Empowers Multilingual LLM Judges

Mohammad Ghiasvand Mohammadkhani, Hamid Beigy

机构 * Amirkabir University of Technology（阿姆irkabir技术大学） ； Sharif University of Technology（沙里夫技术大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

A Review of LLM-Assisted Ideation

Sitong Li, Stefano Padilla, Pierre Le Bras, Junyu Dong, Mike Chantler

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?

Muntasir Wahed, Xiaona Zhou, Kiet A. Nguyen, Tianjiao Yu, Nirav Diwan, Gang Wang, Dilek Hakkani-Tür, Ismini Lourentzou

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Winner Defender Team at Amazon Nova AI Challenge 2025

Generative AI in Agriculture: Creating Image Datasets Using DALL.E's Advanced Large Language Model Capabilities

Ranjan Sapkota, Manoj Karkee

专题命中 评测与基准 ：large language model(title)；language model(title)

A Gold Standard Dataset and Evaluation Framework for Depression Detection and Explanation in Social Media using LLMs

Prajval Bolegave, Pushpak Bhattacharya

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation

Zhongzhen Wen, Yinghui Zhang, Zhong Li, Zhongxin Liu, Linna Xie, Tian Zhang

机构 * State Key Lab for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学） ； The State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Grid-LOGAT: Grid Based Local and Global Area Transcription for Video Question Answering

Md Intisar Chowdhury, Kittinun Aukkapinyo, Hiroshi Fujimura, Joo Ann Woo, Wasu Wasusatein, Fadoua Ghourabi

机构 * AWL, Inc.（AWL公司） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Copyright 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models

Kuofeng Gao, Shu-Tao Xia, Ke Xu, Philip Torr, Jindong Gu

机构 * Tsinghua University（清华大学） ； University of Oxford（牛津大学） ； Peng Cheng Laboratory（鹏城实验室） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2025

Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation

Jiaju Chen, Yuxuan Lu, Xiaojie Wang, Huimin Zeng, Jing Huang, Jiri Gesi, Ying Xu, Bingsheng Yao, Dakuo Wang

机构 * Rice University（里士满大学） ； Northeastern University（东北大学） ； Amazon（亚马逊） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Harvard University（哈佛大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models

Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu

机构 * Hangzhou Institute for Advanced Study（杭州先进研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.AI

Comments Appendix at: https://github.com/PAMPAS-Lab/ANA-PPT-Anamation/blob/main/Appendix.pdf

NbBench: Benchmarking Language Models for Comprehensive Nanobody Tasks

Yiming Zhang, Koji Tsuda

机构 * Department of Computational Biology and Medical Sciences（计算生物学与医学科学系） ； The University of Tokyo（东京大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.LG

Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation

Minju Kim, Dongje Yoo, Yeonjun Hwang, Minseok Kang, Namyoung Kim, Minju Gwak, Beong-woo Kwak, Hyungjoo Chae, Harim Kim, Yunjoong Lee, Min Hee Kim, Dayi Jung, Kyong-Mee Chung, Jinyoung Yeo

机构 * Yonsei University（延世大学） ； Eulji University（欧斯吉大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI

Comments Published at ACL 2025 Findings

Evaluating the Promise and Pitfalls of LLMs in Hiring Decisions

Eitan Anzenberg, Arunava Samajpati, Sivasankaran Chandrasekar, Varun Kacholia

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 10 pages, 2 figures, 2 tables. Submitted to NeurIPS 2025

MIMII-Agent: Leveraging LLMs with Function Calling for Relative Evaluation of Anomalous Sound Detection

Harsh Purohit, Tomoya Nishida, Kota Dohi, Takashi Endo, Yohei Kawaguchi

机构 * Hitachi Ltd., R\&D Group, Tokyo, Japan 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents

Ivoline Ngong, Swanand Kadhe, Hao Wang, Keerthiram Murugesan, Justin D. Weisz, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy

机构 * IBM Research（IBM研究院） ； University of Vermont（佛罗里达大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 2 figures

AgroBench: Vision-Language Model Benchmark in Agriculture

Risa Shinoda, Nakamasa Inoue, Hirokatsu Kataoka, Masaki Onishi, Yoshitaka Ushiku

专题命中 评测与基准 ：language model(title,abstract)

Comments ICCV 2025

SAViL-Det: Semantic-Aware Vision-Language Model for Multi-Script Text Detection

Mohammed-En-Nadhir Zighem, Abdenour Hadid

机构 * Sorbonne Center for Artificial Intelligence, Sorbonne University Abu Dhabi, UAE（索邦人工智能中心，阿布扎比分校，阿联酋） 纠错

专题命中 评测与基准 ：language model(title,abstract)

M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision

Kailai Zhou, Fuqiang Yang, Shixian Wang, Bihan Wen, Chongde Zi, Linsen Chen, Qiu Shen, Xun Cao

专题命中 评测与基准 ：foundation model(title,abstract)

Comments accepted by ICCV2025

Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text

Mizanur Rahman, Md Tahmid Rahman Laskar, Shafiq Joty, Enamul Hoque

机构 * York University（约克大学） ； Dialpad ； Salesforce AI Research（Salesforce人工智能研究） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Vuyko Mistral: Adapting LLMs for Low-Resource Dialectal Translation

Roman Kyslyi, Yuliia Maksymiuk, Ihor Pysmennyi

机构 * National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"（乌克兰国家技术大学 "伊戈尔·西科斯基基土夫技术学院"） ； Ukrainian Catholic University（乌克兰天主教大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Preprint. Will be published at Proceedings of the Fourth Ukrainian Natural Language Processing Workshop (UNLP)

Journal ref Proceedings of the Workshop on Ukrainian Natural Language Processing (UNLP), ACL 2025

Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs

Elisa Forcada Rodríguez, Olatz Perez-de-Viñaspre, Jon Ander Campos, Dietrich Klakow, Vagrant Gautam

机构 * Erasmus Mundus Master in Language and Communication Technologies（埃拉斯谟 Mundus 语言与沟通技术硕士项目） ； HiTZ Center - Ixa, University of the Basque Country (UPV/EHU)（巴斯克国家大学（UPV/EHU）HiTZ中心 - Ixa） ； Cohere（Cohere公司） ； Saarland University（萨尔兰州大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments Workshop on Gender Bias in Natural Language Processing at ACL 2025

Advanced System Integration: Analyzing OpenAPI Chunking for Retrieval-Augmented Generation

Robin D. Pesl, Jerin G. Mathew, Massimo Mecella, Marco Aiello

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments This preprint has not undergone peer review (when applicable) or any post-submission improvements or corrections. The Version of Record of this contribution is published in Advanced Information Systems Engineering. CAiSE 2025. Lecture Notes in Computer Science, vol 15702. Springer, Cham., and is available online at https://doi.org/10.1007/978-3-031-94571-7_8

Journal ref Advanced Information Systems Engineering. CAiSE 2025. Lecture Notes in Computer Science, vol 15702. Springer, Cham

Can LLMs Solve ASP Problems? Insights from a Benchmarking Study (Extended Version)

Lin Ren, Guohui Xiao, Guilin Qi, Yishuai Geng, Haohan Xue

机构 * School of Computer Science and Engineering, Southeast University, Nanjing, China（计算机科学与工程学院，东南大学，南京，中国） ； Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及其跨学科应用关键实验室（东南大学），教育部，中国） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted for publication at the 22nd International Conference on Principles of Knowledge Representation and Reasoning (KR 2025). The code is available at https://github.com/HomuraT/ASPBench

PEMUTA: Pedagogically-Enriched Multi-Granular Undergraduate Thesis Assessment

Jialu Zhang, Qingyang Sun, Qianyi Wang, Weiyi Zhang, Zunjie Xiao, Xiaoqing Zhang, Jianfeng Ren, Jiang Liu

机构 * Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering（可信自主系统研究 institute 和计算机科学与工程系） ； Southern University of Science and Technology（南方科技大学） ； School of Computer Science, University of Nottingham Ningbo China（宁波大学计算机学院） ； School of Ophthalmology and Optometry, Wenzhou Medical University（温州医学院眼视光学院） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI

AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code

Nadeen Fathallah, Daniel Hernández, Steffen Staab

机构 * University of Stuttgart, Artificial Intelligence（斯图加特大学人工智能学院） ； Artificial Intelligence, University of Stuttgart（人工智能学院，斯图加特大学） ； University of Stuttgart（斯图加特大学） ； University of Southampton（南安普顿大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI