语言大模型 / LLM

You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects

Islem Bouzenia, Michael Pradel

机构 * University of Stuttgart（斯图加特大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments PUBLISHED AT ISSTA 2025

Journal ref ISSTA 2025

MODP: Multi Objective Directional Prompting

Aashutosh Nema, Samaksh Gulati, Evangelos Giakoumakis, Bipana Thapaliya

机构 * Dell Technologies（戴尔技术） 纠错

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 5 figures, submission to KDD 2025

Benchmarking LLM-based Relevance Judgment Methods

Negar Arabzadeh, Charles L. A. Clarke

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Leveraging LLM For Synchronizing Information Across Multilingual Tables

Siddharth Khincha, Tushar Kataria, Ankita Anand, Dan Roth, Vivek Gupta

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 17 Pages, 11 Tables, 2 Figures

Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-tasks

Jonathan Salfity, Selma Wanna, Minkyu Choi, Mitch Pryor

专题命中 评测与基准 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 8 pages, 3 figures. IROS 2024 Submission

Evaluating Cultural and Social Awareness of LLM Web Agents

Haoyi Qiu, Alexander R. Fabbri, Divyansh Agarwal, Kung-Hsiang Huang, Sarah Tan, Nanyun Peng, Chien-Sheng Wu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments NAACL 2025 Findings

GroundCocoa: A Benchmark for Evaluating Compositional & Conditional Reasoning in Language Models

Harsh Kohli, Sachin Kumar, Huan Sun

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Comments 16 pages, 17 figures, 3 tables. Accepted to NAACL 2025 (Main)

GUI Agents with Foundation Models: A Comprehensive Survey

Shuai Wang, Weiwen Liu, Jingxuan Chen, Yuqi Zhou, Weinan Gan, Xingshan Zeng, Yuhan Che, Shuai Yu, Xinlong Hao, Kun Shao, Bin Wang, Chuhan Wu, Yasheng Wang, Ruiming Tang, Jianye Hao

专题命中 评测与基准 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

PromptOptMe: Error-Aware Prompt Compression for LLM-based MT Evaluation Metrics

Daniil Larionov, Steffen Eger

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

DRC-Coder: Automated DRC Checker Code Generation Using LLM Autonomous Agent

Chen-Chia Chang, Chia-Tung Ho, Yaguang Li, Yiran Chen, Haoxing Ren

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Proceedings of the 2025 International Symposium on Physical Design (ISPD '25), March 16--19, 2025, Austin, TX, USA

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

Yutao Mou, Shikun Zhang, Wei Ye

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted by NeurIPS2024 (Dataset and Benchmark Track)

LLM-Powered Grapheme-to-Phoneme Conversion: Benchmark and Case Study

Mahta Fetrat Qharabagh, Zahra Dehghanian, Hamid R. Rabiee

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 5 pages, 5 figures

LLM-based multi-agent poetry generation in non-cooperative environments

Ran Zhang, Steffen Eger

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments preprint

Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change Domain

Francesca Grasso, Stefano Locci

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；small language model(abstract)；SLM(abstract)

Comments 11 pages, to be published in NLDB 2024

LAB-Bench: Measuring Capabilities of Language Models for Biology Research

Jon M. Laurent, Joseph D. Janizek, Michael Ruzo, Michaela M. Hinks, Michael J. Hammerling, Siddharth Narayanan, Manvitha Ponnapati, Andrew D. White, Samuel G. Rodriques

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；language agent(abstract)

Comments 40 pages, 5 main figures, 1 main table, 2 supplemental figures, 4 supplemental tables. Submitted to NeurIPS 2024 Datasets and Benchmarks track (in review)

Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting

Jiyue Jiang, Liheng Chen, Sheng Wang, Lingpeng Kong, Yu Li, Chuan Wu

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Identifying Performance-Sensitive Configurations in Software Systems through Code Analysis with LLM Agents

Zehao Wang, Dong Jae Kim, Tse-Hsun Chen

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction

Danyang Zhang, Zhennan Shen, Rui Xie, Situo Zhang, Tianbao Xie, Zihan Zhao, Siyuan Chen, Lu Chen, Hongshen Xu, Ruisheng Cao, Kai Yu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing

Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, Elena Glassman

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 18 pages, 7 figures, published at CHI 2024

SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents

Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk, Hao Zhu

专题命中 评测与基准 ：language agent(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Exploring the Deceptive Power of LLM-Generated Fake News: A Study of Real-World Detection Challenges

Yanshen Sun, Jianfeng He, Limeng Cui, Shuo Lei, Chang-Tien Lu

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Multi-Query Focused Disaster Summarization via Instruction-Based Prompting

Philipp Seeberger, Korbinian Riedhammer

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments CrisisFACTS (TREC 2023)

CoAScore: Chain-of-Aspects Prompting for NLG Evaluation

Peiyuan Gong, Jiaxin Mao

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

A Review of Repository Level Prompting for LLMs

Douglas Schonholtz

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 15 figures/charts, 7 pages, Submitted as an NLP project at Northeastern. Focuses on comparing two papers, there are many more papers that could be included

Prompting with Pseudo-Code Instructions

Mayank Mishra, Prince Kumar, Riyaz Bhat, Rudra Murthy, Danish Contractor, Srikanth Tamilselvam

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Published in EMNLP 2023 main track

A ML-LLM pairing for better code comment classification

Hanna Abi Akl

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 10 pages, 2 figures, 2 tables, accepted for the Information Retrieval in Software Engineering track at the Forum for Information Retrieval Evaluation 2023

Journal ref Information Retrieval in Software Engineering track at Forum for Information Retrieval Evaluation 2023

Improved prompting and process for writing user personas with LLMs, using qualitative interviews: Capturing behaviour and personality traits of users

Stefano De Paoli

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Safurai 001: New Qualitative Approach for Code LLM Evaluation

Davide Cifarelli, Leonardo Boiardi, Alessandro Puppo

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments 22 pages, 1 figure, 3 tables

Approximating Online Human Evaluation of Social Chatbots with Prompting

Ekaterina Svikhnushina, Pearl Pu

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments accepted to SIGDIAL 2023 (long paper)

Short Answer Grading Using One-shot Prompting and Text Similarity Scoring Model

Su-Youn Yoon

专题命中 评测与基准 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 7 pages, 2 figures